DIGITAL HOLOGRAPHY AND

DIGITAL IMAGE PROCESSING:

Principles, Methods, Algorithms
 DIGITAL HOLOGRAPHY AND
DIGITAL IMAGE PROCESSING:
Principles, Methods, Algorithms

by

Leonid Yaroslavsky
Tel Aviv University, Israel

SPRINGER SCIENCE+BUSINESS MEDIA, LLC

Library of Congress Cataloging-in-Publication

CIP info or:

Title: Digital Holography and Digital Image Processing: Principles, Methods, Algorithms
Author (s): Leonid Yaroslavsky
ISBN 978-1-4419-5397-1 ISBN 978-1-4757-4988-5 (eBook)
DOI 10.1007/978-1-4757-4988-5

Copyright © 2004 by Springer Science+Business Media New York

Originally published by Kluwer Academic Publishers in 2004
Softcover reprint ofthe hardcover Ist edition 2004
All rights reserved. No part of this publication may be reproduced, stored in a retrieval
system or transmitted in any form or by any means, electronic, mechanical, photo-copying,
microfilming, recording, or otherwise, without the prior written permis sion of the publisher,
with the exception of any material supplied specifically for the purpose of being entered and
executed on a computer system, for exclusive use by the purchaser of the work.
Permissions forbooks published in the USA: permissions@wkap. corn
Permissions for books published in Europe: [email protected]
Printed on acid-free paper.
 TABLE OF CONTENTS

1. INTRODUCTION .....•..........................•................................. 1

1.1 DIGIT AL HOLOGRAPHY AND EVOLUTION OF IMAGING

TECHNIQUES .•.••.....•.. , •.••..•..........•••. , '" ..•.•.......•......•....•.............. 1

1.2 CONTENTS OF THIS DOOK ..•.....•.•.........•..........•...•......... 5

REFERENCES .....•.....................•........•...........•..................... 10

2. OPTICAL SIGNALS AND TRANSFORMS •.•.•.•.......•..•............ 11

2.1 MATHEMA TICAL MODELS OF OPTICAL SIGNALS ....•... , ..•... 11

2.1.1 Primary definitions and classification ................................................. 11
2.1.2 Signal space ........................................................................................ 13
2.1.3 Linear signal space, basis functions and signal representation as
expansion over a set ofbasis functions ................................................................ 17
2.1.4 Integral representation of signals ........................................................ 22

2.2 SIGNAL TRANSFORMATIONS .......................................... 24

2.3IMAGING SYSTEMS AND INTEGRAL TRANSFORMS ...........•. 27

2.3.1 Direct imaging: Convolution integral ................................................. 27
2.3.2 Imaging in Fourier domain: Holography and diffraction integrals ..... 30

2.4 FOURIER TRANS FORM AND ITS DERIVATIVES ................. 39

2.4.1 Properties of integral Fourier Transfonn ............................................ 39
2.4.2 Special cases ofthe integral Fourier transfonn: integral Cosine,
Hartley, Hankel and Mellin transfonns ................................................................ 44
2.4.3 Integral Fresnel and related transfonn ................................................ 49
2.4.4 Hilbert transfonn ................................................................................ 53

2.5IMAGING FROM PROJECTIONS: RADON AND ADEL

TRANSFORMS .............................................................................. 57

2.6 MULTI RESOLUTION IMAGING: W AVELET TRANSFORMS ... 61

 2.7SLIDING WINDOW TRANSFORMS AND "TIME-FREQUENCY"
(SPACE-TRANSFORM) SIGNAL REPRESENTA TION •..............•..•..... 63
2.7.1 Transforms in sliding window (windowed transforms) 63
2.7.2 Sliding window Fourier Transform .................................................... 53
2.7.3 Sliding window and wavelet transforms as signal sub-band
decomposition ...................................................................................................... 64

2.8 STOCHASTIC TRANSFORMATIONS AND STA TISTICAL

MODELS ...........•.••..............••.•.........•..........•.•••.••.•.............•..•..... 67
2.8.1 Additive, multiplicative, Poisson noise and impulse noise models .... 67
2.8.2 Speckle noise model ........................................................................... 72

References .............................................................................. 77

3. DIGITAL REPRESENT AT ION OF SIGNALS ............................ 79

3.1 PRINCIPLES OF SIGNAL DIGITIZA TION ........................... 79

3.2 SIGNAL DISCRETIZATION AS EXPANSION OVER A SET OF

BASIS FUNCTIONS. TYPICAL BASIS FUNCTIONS AND
CLASSIFICATION ...................................................................... 80
3.2.1 Shift (convolution) basis functions ..................................................... 81
3.2.2 Scale (multiplicative) basis functions ................................................. 85
3.2.3 Wavelets ............................................................................................. 92
3.2.4 Discrete bases ..................................................................................... 94
3.2.5 Optimality ofbases. Karhunen-Loeve, Hotelling and Singular Values
Decomposition Transforms .................................................................................. 97
3.2.6 Two dimensional and multi dimensional bases ................................ 103

3.3 SHIFT (CONVOLUTION) BASES FUNCTIONS AND SAMPLING

THEOREM ............................................................................... 106
3.3.1 1-D samp ling theorem ...................................................................... 106
3.3.2 Sampling 2-D and multi dimensional signals .................................... 113
3.3.3 Sampling artifacts: qualitative analysis ............................................ 120
3.3.4 Sampling artifacts: quantitative analysis .......................................... 124

3.4 MULTI-RESOLUTION SAMPLING .................................... 129

3.5UNCONVENTIONAL DIGITAL IMAGING METHODS ............ 131

3.6PRINCIPLES OF SIGNAL SCALAR QUANTIZATION ............ 133

3.6.1 Optimal homogeneous non-uniform quantization ............................ 133
 3.6.2 Quantization in digital holography ................................................... 143

3.7BASICS OF SIGNAL CODING AND DATA COMPRESSION ..... 147

3. I.l Signal rate distortion funetion, entropy and statistical eoding ......... 147
3.7.2 Image eompression methods: a review ............................................. 152

4. DIGITAL REPRESENTATION OF SIGNAL

TRANSFORMATIONS ...........•..... 161

4.1 THE PRINCIPLES .......................•...••............................. 161

4.2 DISCRETE REPRESENT ATION OF CONVOLUTION INTEGRAL.

DIGITAL FILTERS ..................................................................... 170

4.3 DISCRETE REPRESENT ATION OF FOURIER INTEGRAL

TRANS FORM.•.................•.............•...•....................................... 170
4.3.1 Diserete Fourier Transforms ............................................................... 170
4.3.2 Properties of Diserete Fourier Transforms .......................................... 175
4.3.3 Diserete eosine and sine transforms .................................................... 189
4.3.4 Lapped Transforms ............................................................................. 195

4.4 DlSCRETE REPRESENT ATION OF FRESNEL INTEGRAL

TRANSFORM ............................................................................ 199
4.4. I Diserete Fresnel Transform ................................................................. 199
4.4.2 Properties of Diserete Fresnel Transforms .......................................... 205

References ............................................................................ 209

5. METHODS AND ALGORITHMS OF DIGITAL FILTERING ....... 211

5.1 FILTERING IN SIGNAL DOMAIN ............•.•..................... 211

5.1.1 Transversal and reeursive filtering ...................................................... 211
5.1.2 Separable, easeade, paraBel and paraBel reeursive filtering ................ 213
5.1.3 Reeursive algorithms ofsliding window DFT and DCTlDeST
................................................................................. 220
5.1.4 Border proeessing in digital filtering ................................................... 223

5.2 FILTERING IN TRANSFORM DOMAIN .............................. 226

5.2.1 DFT and eyelie eonvolution. Border proeessing ................................. 226
5.2.2 Signal eonvolution in the DCT domain ............................................... 227
 5.3 COMBINED ALGORITHMS FOR COMPUTING DFT AND DCT
OF REAL VALUED SIGNALS .•.•.•...........••........•...•.....•...............• 231
5.3.1 Combined algorithms for computing DFT of signals with real sampies
... '" ........................................................................... 231
5.3.2 Combined algorithm for computing the DCT via FFT ....................... 234

References ......••..........•.•....................•.............•....•............... 237

6. FAST ALGORITHMS .................•...............•.••.................... 239

6.1 THE PRINCIPLE OF FAST FOURIER TRANSFORMS ............ 239

6.2 MATRIX TECHNIQUES IN FAST TRANSFORMS ................... 243

6.3 TRANSFORMS AND THEIR FAST ALGORITHMS IN MATRIX

REPRESENT AT ION ..................................................................... 249
6.3.1 Walsh-Hadamard transform ................................................................ 249
6.3.2 Walsh-Paley Transform and Bit Reversal Permutation ....................... 250
6.3.3 Walsh Transform and Binary-to-Gray Code and Hadamard
Permutations ...................................................................................................... 252
6.3.4 Haar Transform ................................................................................... 254
6.3.5 Discrete Fourier Transform ................................................................. 255
6.3.6 DCT and DcST ................................................................................... 260

6.4 PRUNED ALGORITHMS ................................................... 264

6.5 QUANTIZED DFT .......................................................... 269

REFERENCES ..................................................................... 274

7. STATISTICAL METHODS AND ALGORITHMS ...................... 275

7.1 MEASURING SIGNAL STATISTICAL CHARACTERISTICS .... 275

7.1.1 Measuring probability distribution and its moments and order
statistics ................................................................................. 275
7.1.2 Measuring signal corre1ation functions and spectra ...................... 280
7.1.3 Measuring parameters of random interferences in sensor and imaging
systems ............................ , .................................................... 284

7.2 DIGITAL STATISTICAL MODELS AND MONTE CARLO

METHODS ............................................................................... 291
 7.2.1 Principles of generating pseudo-random numbers. Generating
independent uniformly distributed pseudo-random numbers ............................. 291
7.2.2 Generating pseudo-random numbers with special statistical properties
................................................................................. 292
7.2.3 Generating pseudo-random images ..................................................... 297
7.2.4 Generating correlated phase masks. Kinoform and programmed
diffuser holograms ............................................................................................. 300

7.3 STATISTICAL (MONTE CARLO) SIMULATION. CASE STUDY:

SPECKLE NOISE PHENOMENA IN COHERENT IMAGING AND
DIGITAL HOLOGRAPHY ..........................................................305
7.3.1 Computer model .................................................................................. 306
7.3.2 Simulation results ................................................................................ 307

References ............................................................................312

8. SENSOR SIGNAL PERFECTING, IMAGE RESTORA TION,

RECONSTRUCTION AND ENHANCEMENT .................................... 313

8.1 MATHEMATICAL MODELS OF IMAGING SYSTEMS ............313

8.2 LINEAR FILTERS FOR IMAGE RESTORATION................... 315

8.2.1 Transform domain MSE optimal scalar Wiener Filters ....................... 315
8.2.2 Empirical Wiener filters for image denoising ..................................... 317
8.2.3 Image deblurring, inverse filters and aperture correction .................... 323

8.3 SLIDING WINDOW TRANS FORM DOMAIN ADAPTIVE SIGNAL

RESTORA TION ........................................................................ 328
8.3.1 Local adaptive filtering ....................................................................... 328
8.3.2 Sliding window transform domain DCT filtering ............................... 330
8.3.3 Hybrid DCT/wavelet filtering ............................................................. 334

8.4 MULTI-COMPONENT IMAGE RESTORA TION .....................340

8.5 FILTERING IMPULSE NOISE ............................................343

8.6 METHODS FOR CORRECTING GRAY SCALE NONLINEAR

DISTORTIONS ............................................................................348

8.7 IMAGE RECONSTRUCTION ............................................353

 8.8 IMAGE ENHANCEMENT ................................•................•. 361
8.8.1 Image enhaneement as an image proeessing task. Classifieation of
image enhaneement methods ............................................................................. 361
8.8.2 Gray level histogram modifieation methods ....................................... 362
8.8.3 Image speetra modifieation methods ................................................... 367
8.8.4 Using color, stereo and dynamieal vision for image enhaneement ... 369

References ............................................................................371

9. IMAGE RESAMPLING AND GEOMETRICAL

TRANSFORMATIONS .................................................................. 373

9.1 PRINCIPLES OF IMAGE RESAMPLING .............................. 373

9.2 NEAREST NEIGHBOR, LINEAR AND SPLINE INTERPOLATION

METHODS ................................................................................376

9.3 ALGORITHMS OF DISCRETE SINC-INTERPOLA TION .........380

9.3.1 Diserete sine-interpolation by zero padding signal OFT speetrum .. 380
9.3.2 OFT based diserete sine-interpolation algorithm for signal arbitrary
translation ........................................................................................................... 382
9.3.3 DCT based diserete sine-interpolation algorithm for signal arbitrary
translation ................................................................................. 386
9.3.4 Sliding window adaptive diserete sine-interpolation algorithms
................................................................................. .388

9.4 Application examples ........................................................... 395

9.4.1 Signal and image resizing and loealization with sub-pixel aeeuraey 395
9.4.2 Fast algorithm for image rotation with diserete-sine-interpolation .. 398
9.4.3 Signal differentiating and filtered back projeetion method for
tomographie reeonstruetion ................................................................................ 400
9.4.4 Signal integrating and reeonstrueting surfaees from their slope ...... .403
9.4.5 Polar-to-Cartesian eoordinate eonversion, interpolation of OFT speetra
ofprojeetions and the direet Fourier method ofimage reeonstruetion from
projeetions .......................................................................................................... 407

References ............................................................................ 410

 10. SIGNAL PARAMETER ESTIMATION AND MEASUREMENT.
OBJECT LOCALIZATION ...........•.................................•.............. 411

10.1 PROBLEM FORMULATION. OPTIMAL STATISTICAL

ESTIMA TES •.•...............•.........................................•...•...•........... 411

10.2 LOCALIZATION OF AN OBJECT IN THE PRESENCE OF

ADDITIVE WHITE GAUSSIAN NOISE ..•....•..•......•.......................... 414
10.2.1 Optimallocalization device. Correlator and matched filter ...... .414
10.2.2 Computer and optical implementations ofthe matched filter
correlator ............................................................................. .417
10.2.3 Performance of optimal estimators: normal and anomalous
localization errors ............................................................................................... 419

10.3 PERFORMANCE OF THE OPTIMAL LOCALIZA TION DEVICE

...............•.....•......•...•............•....•..........................•.•.............. 421
10.3.1 Distribution density and variance ofnormal errors ..................... .421
10.3.2 Illustrative examples .................................................................... 431
10.3.3 Localization accuracy for non-optimallocalization devices ..... .436
10.3.4 Localization reliability. Probability ofanomalous localization errors
............................................................................. .439

10.4 LOCALIZATION OF AN OBJECT IN TUE PRESENCE OF

ADDITIVE CORRELATED GAUSSIAN NOISE ................................ .443
10.4.1 Localization of a target object in the presence of non-white
(correlated) additive Gaussian noise .................................................................. 443
10.4.2 Localization accuracy ofthe optimal filter .......................... .445

10.5 OPTIMAL LOCALIZA TION IN COLOR AND MULTI

COMPONENT IMAGES •.......................•..•.................................. 449
10.5.1 Optimallocalization device .......................................................... 449
10.5.2 Localization accuracy and reliability ................................. .451
10.5.3 Optimallocalization in multi component images with correlated
noise. . .......................................................................... 452

10.6 OBJECT LOCALIZATION IN THE PRESENCE OF MULTIPLE

NONOVERLAPPNING NON-TARGET OBJECTS .•...•........................ 454

References ................•..•....................•.•.•...••.•.•...................... 459

 11. TARGET LOCATION IN CLUTTER ....•.............................. .461

11.1 PROBLEM FORMULATION ••.•..•.........•.......................•.• .461

11.2 LOCALIZATION OF PRECISELY KNOWN OBJECTS:

SPATIALLY HOMOGENEOUS OPTIMALITY CRITERION ......•........ 465
11.2.1 Optimal adaptive correlator .......................................................... 465
11.2.2 Implementation issues, practical recommendations and illustrative
examples ............................................................................. .469

11.3 LOCALIZATION OF INEXACTLY KNOWN OBJECT:

SPATIALLY HOMOGENEOUS CRITERION .................................... 474

11.4 LOCALIZATION METHODS FOR SPATIALLY

INHOMOGENEOUS CRITERIA ................................................... 478

11.5 OBJECT LOCALIZATION AND IMAGE BLUR ................... 483

11.6 OBJECT LOCALIZATION AND EDGE DETECTION.

SELECTION OF REFERENCE OBJECTS FOR TARGET TRACKING
..............................................................................................485

11.7 OPTIMAL ADAPTIVE CORRELATOR AND OPTICAL

CORRELATORS ........................................................................ 490
11.7.1 Optical correlators with improved discrimination capability
............................................................................. .490
11.7.2 Computer and optical implementations ofthe optimal adaptive
correlator ............................................................................. .493

11.8 TARGET LOCATING IN COLOR AND MULTI COMPONENT

IMAGES ................................................................................... 499
11.8.1 Theoretical framework ................................................................. 499
11.8.2 Separable component-wise implementations ofthe optimal adaptive
multi-component correlator................................................................................ 501

References ............................................................................ 507

12. NON LINEAR FILTERS IN SIGNAL/IMAGE PROCESSING ...... 509

12.1 CLASIFICATION PRINCIPLES ........................................ 510

12.1.1 Main assumptions and definitions ................................................ 510
 12.1.2 Typical signal attributes ............................................................... 511
12.1.3 Estimation operations ................................................................... 512
12.1.4 Neighborhood building operations ............................................... 515

12.2 FILTER CLASSIFICATION TABLES ...•.......•..................... 519

12.2.1 Transferential filters ..................................................................... 519
12.2.2 Iterative filtering ........................................................................... 522
12.2.3 Multiple branch parallel, cascade and recursive filters ................ 523
12.2.4 Miscellaneous new filters ............................................................. 524

12.3 PRACTICAL EXAMPLES ....•.....•.•.....•..•....•.................•.•. 527

12.3.1 Image denoising ........................................................................... 527
12.3.2 Image enhancement. ..................................................................... 531
12.3.3 Image segmentation and edge detection ....................................... 535
12.3.4 Implementation issues .................................................................. 538

References ............................................................................ 540

13. COMPUTER GENERATED HOLOGRAMS ........................... 541

13.1 MATHEMATICAL MODELS ............................................. 541

13.2 METHODS FOR ENCODING AND RECORDING COMPUTER

GENERATED HOLOGRAMS .......................................................... 548

13.3 RECONSTRUCTION OF COMPUTER GENERATED

HOLOGRAMS '" ........................................................................ 565

References ............................................................................ 575

 Chapter 1

INTRODUCTION

1.1 DIGITAL HOLOGRAPHY AND EVOLUTION OF

IMAGING TECHNIQUES

The history of science is, to a considerable degree, the history of

invention, development and perfecting imaging methods and devices.
Modem science began with the invention and application of optical
telescope and microscope in the beginning of 17-th century by Galileo
Galilei (1564-1642), Anthony Leeuwenhoek (1632-1723) and Robert Hook
(1635-1703).
The next decisive stage was invention of photography in the first half of
the 19-th century. Photographic plates were the major means in discoveries
of X-rays by Wilhelm Conrad Roentgen (1845-1923, The Nobel Prize
Laureate, 1901) and radio activity by Antoine Henri Becquerel (1852-1908,
The Nobel Prize Laureate, 1903) at the end of 19-th century. These
discoveries, in their turn, almost immediately gave birth to new imaging
techniques: to X-ray imaging and radiography.
X-rays were discovered by W.C. Roentgen in experiments with cathode
rays. Cathode rays were discovered by Julius Plücker (1801-1868) in 1859
who used vacuum tubes invented in 1855 by German inventor Heinrich
Geissler (1815-1879). These tubes, as modified by Sir William Crookes
(1832-1919) led eventually to the discovery of the electron and finally
brought about the development of electronic television and electron
microscopy in 30-th - 40-th of 20-th century. Designer of the first electron
microscope E. Ruska (1906-1988) was awarded The Nobel Prize in Physics
for 1986. This award was shared with G. Binning and H. Rohrer, who were
awarded for their design ofthe scanning tunneling microscope.
2 Chapter 1

The discovery ofdiffraction ofX-rays by Max von Laue (1889-1960, the

Nobel Prize Laureate, 1914) in the beginning of 20-th century marked the
advent of a new imaging technique, imaging in transfonn domain. Although
Von Laue's motivation was not creating a new imaging technique but rather
proving the wave nature of X-rays, lauegrams had very soon become the
main imaging tool in crystallography. Lauegrams are not conventional
images for visual observation. However, using lauegrams, one could
numerically reconstruct the spatial structure of atoms in crystals. This shoot
gave its crop in less then half a century. One of the most remarkable
scientific achievements of20-th century is based on X-ray crystallography.1t
is discovery by J. Watson and F. Crick ofspiral structure ofDNA (the Nobel
Prize, 1953). And about the same time a whole bunch of such new transfonn
domain imaging methods had appeared: holography, synthetic aperture
radar, coded aperture imaging, tomography. Two of these inventions were
awarded the Nobel Prize: D. Gabor for "his invention and developments of
the holographie method" (the Nobel Prize in Physics, 1971) and A. M.
Connack and G. N. Hounsfield for "the development of computer assisted
tomography" (the Nobel Prize in Physiology and Medicine, 1979).
Denis Gabor invented holography in 1948 ([1]). This is what D. Gabor
wrote in his Nobel Lecture ([2]) about the development of holography: "
Around 1955 holography went into a long hibernation. The revival came
suddenly and explosively in 1963, with the publication of first successful
laser hologram by Emmett N. Leith and Juris Upatnieks ofthe University of
Michigan, An Arbor ([3]). Their success was due not only to the laser, but to
the long theoretical preparation of Emmett Leith (in the field of the "side
looking radar") which started in 1955 ... Another important development in
holography [happened] in 1962, just before the "holography explosion".
Soviet physicist Yu. N. Denisyuk published an important paper ([4]) in
which he combined holography with the ingenious method of photography
in natural colors, for which Gabriel Lippman received the Nobel Prize in
1908".
Denis Gabor received his Nobel Prize in 1971. The same year, a paper
"Digital holography" was published in Proceedings of IEEE by T. Huang
([5]). This paper marked the next step in the development ofholography, the
use of digital computers for reconstructing, generating and simulating wave
fields, and reviewed pioneer accomplishments in this field. These
accomplishments prompted a burst of research and publications in early and
mid 70-th. At that time, most of the main ideas of digital holography were
suggested and tested ([6-16]). Numerous potential applications of digital
holography such as fabricating computer generated diffractive optical
elements and spatial filters for optieal infonnation proeessing, 3-D
holographie displays and holographie television, holographie computer
vision stimulated a great enthusiasm among researchers.
 1. Introduction 3

However, limited speed and memory capacity of computers available at

that time, absence of electronie means and media for sensing and recording
optical holograms hampered implementation of these potentials. In 1980-th
digital holography went into a sort of hibernation similarly to what happened
to holography in 1950-th - 1960-th. With an advent, in the end of 1990-th, of
the new generation of high speed microprocessors, high resolution electronic
optical sensors and liquid crystal displays, of a technology for fabricating
miero lens and mirror arrays digital holography is getting a new wind.
Digital holography tasks that required hours and days of computer time in
1970-th can now be solved in almost "real" time for tiny fractions of
seconds. Optical holograms can now be directly sensed by high resolution
photo electronic sensors and fed into computers in "real" time with no need
for any wet photo-chemical processing. Micro lens and mirror arrays
promise a breakthrough in the means for recording computer generated
holograms and creating holographic displays. Recent flow of publications in
digital holographic metrology and microscopy indicate revival of digital
holography from the hibernation§.
The development of optical holography, one of the most remarkable
inventions of the XX-th century, was driven by c1car understanding of
information nature of optics and holography ([ 17-19]). The information
nature of optics and holography is especially distinctly seen in digital
holography. Wave field recorded in the form of a hologram in optieal, radio
frequency or acoustic holography, in digital holography is represented by a
digital signal that carries the wave field information deprived of its physical
casing. With digital holography and with incorporating digital computers
into optical information systems, information optics has reached its maturity.
The most substantial advantage of digital computers as compared with
analog electronic and optical information processing devices is that no
hardware modifications are necessary to solving different tasks. With the
same hardware, one can build an arbitrary problem solver by simply
selecting or designing an appropriate code for the computer. This feature
makes digital computers also an ideal vehicle for processing optical signals
adaptively since, with the help of computers, they can adapt rapidly and
easily to varying signals, tasks and end user requirements. In addition,
acquiring and processing quantitative data contained in optical signals, and
connecting optical systems to other informational systems and networks is
most natural when data are represented and handled in a digital form. In the
same way as monies are the general equivalent in economics, digital signals
are the general equivalent in information handling. Thanks to its universal

§ Curiously enough that the initial meaning of digital holography is getting forgotten and, in
many recent publications, digital holography is associated only with numerical
reconstruction ofholograms .
4 Chapter 1

nature, the digital signal is an ideal means for integrating different

informational systems.
This is not a coincidence that digital holography appeared in the end of
60-th, the same period of time to which digital image processing can be
dated back to. In the same way, in a certain sense, as inventing by Ch.
Towns, G. Basov and A. Prokhorov lasers in mid 1950-th (the Nobel Prize in
Physics, 1964) stimulated development ofholography, two events stimulated
digital holography and digital image processing: beginning of industrial
production of computers in 1960-th and introducing Fast Fourier Transform
algorithm made by J. W. Cooleyand J. M. Tukey in 1965 ([21]).
Digital holography and digital image processing are twins. They share
common origin, common theoretical base, common methods and algorithms.
This is the purpose of the present book to describe these common principles,
methods and algorithms for senior-level undergraduate and graduate
students, researchers and engineers in optics, photonics, opto-electronics and
electronic engineering.
 1. Introduction 5

1.2 CONTENTS OF THIS BOOK

The theoretical base for digital holography and image processing is signal
theory. Ch. 2 introduces basic concepts of the signal theory and
mathematical models that are used for describing optical signals and imaging
systems. The most important models are integral transforms. All major
integral transforms, their properties and interrelations are reviewed:
convolution integral; Fourier transform and such its derivatives as Cosine,
Hartley, Hankel, Mellinn transforms; Fresnel transform; Hilbert transform;
Radon and Abel transforms; wavelet transforms; sliding window transforms.
In the last section of the chapter, stochastic signal transformations and
corresponding statistical models are introduced.
One of the most fundamental problems of digital holography and, more
generally, of integrating digital computers and analog optics is that of
adequate representation of optical signal and transformations in digital
computers. Solving this problem requires consistent account for computer-
to-optics interface and for computational complexity issues. These problems
are treated in Chs. 3 , for signals, and in Ch.4, for signal transforms.
In Ch. 3, general signal digitization principles and concepts of signal
discretization and element-wise quantization as a practical way to implement
the digitization are introduced. Signal discretization is treated as signal
expansion over a set of discretization basis functions and classes of shift,
scale and combined shift-scale basis function are introduced to describe
traditional signal sampling and other image discretization techniques
implemented in coded aperture imaging, synthetic aperture imaging,
computer and MRI tomography. Naturally, image sampling theory, including
I-D and 2-D sampling theorems and analysis of sampling artifacts, is treated
at the greatest length. Treatment of element-wise quantization is mostly
focused on the compander - expander optimal non-uniform quantization
method that is less covered in the literature then Max-Lloyd numerical
optimization of the quantization. Aseparate section is devoted to
peculiarities of signal quantization in digital holography. The chapter
concludes with a review of image compression methods. As this subject is
very weIl covered in the literature, only basic principles of data compression
and a classification and brief review of the methods are offered.
Ch. 4 provides a comprehensive exposure of discrete representation of
signal transforms in computers. In digital holography, computers should be
considered as an integral part of optical systems. This requires observing
mutual correspondence principle between discrete signal transforms in
computers and analog transforms in optical system that is formulated in Sect.
4.1. On the base of this principle, discrete representations of the convolution
integral, Fourier and Fresnel integral transforms are developed in Sects. 4.2 -
4.4. Conventional Discrete Fourier and Discrete Fresnel Transforms are
represented here in a more general form that takes into consideration that
 6 Chapter 1

image or hologram sampling and reeonstruetion deviees can be placed in

optical setups with arbitrary shifts with respect to their optieal axes. This
requires introdueing into discrete Fourier and Fresnel transforms arbitrary
shift parameters. The presence of these parameters provides to modified in
this way transforms some new useful features such as flexibility of image
resampling using Oiscrete Fourier Transform (OFT) and allows to treat
numerous modifications of OFT and, primarily, Oiscrete Cosine Transform
(OCT), in the most natural way as special eases of OFT for different shift
parameters and different type of signal symmetry.
A critical issue in digital holography and image proeessing is the
computational complexity of the processing. The availability of fast
computation algorithm determine feasibility of their practical applications.
These issues are treated in Chs. 5 and 6.
Ch. 5 deseribes efficient computational algorithms for image and
hologram filtering in signal domain. Oescribed are separable, recursive,
parallel and cascade implementations of digital filters; new implementation
of signal convolution in OCT domain that is practically free of boundary
effects of the traditional cyclie convolution with OFT; recursive filtering in
sliding window in the domain of OCT and other transforms; combined
algorithms of 1-0 and 2-0 OFT and OCT for real valued signals.
Ch. 6 exposes basic principles of the so called fast trans form of which
Fast Fourier Transform is the most known special case. The fast transforms
are treated here in the most general matrix form that allows to formalize their
design and provides a very compact and general formulas for the fast
algorithms. From the practieal point of view, pruned algorithms and
quantized OFT described in this chapter may represent the most interest for
the reader.
Many phenomena in holography and, generally ,in imaging are treated as
random, stochastic. In addition, one of the task of digital holography is
statistical simulation of holographie and imaging processes. Hence, the next
chapter, Ch. 7, deals with statistical methods and algorithms. It covers
basically all involved issues, from measuring signal statistical characteristics
for different statistical models of signals to building digital statistical models
and to generating arrays of pseudo-random numbers with prescribed
statistical characteristics. Practical applications are represented in this
chapter by methods for generating correlated phase masks and diffusers and
by demonstration of statistical simulation for studying speckle noise
phenomena in imaging systems that use coherent radiation.
The primary task of processing images and holograms after they are
digitized and put into computer is usually correcting distortions in imaging
and holographie systems that prevent images from being perfect for the end
user. We call this task sensor signal perfection. In image processing, the term
image restoration is commonly accepted to designate this task. Image
reconstruction in holography and tomography mayaIso be regarded a similar
 1. Introduction 7

task. Methods for sensor signal perfection, image restoration and image
reconstruction are discussed in Ch. 8. The methods assurne a canonical
imaging system model, described in Sect. 8.1, that consist of a cascade of
units that perform signal linear transformation, nonlinear point-wise
transformation and stochastic transformation such as adding to the signal
signal independent noise. As a theoretical bench mark, the design of optimal
linear filters is used. The filters correct signal distortions in the linear
filtering units of the system model such as image blur, and suppress additive
signal independent noise in such a way as to minimize root mean square
restoration error measured over either a set of images in an image data base
or image ensemble or even over a particular individual image. In the latter
case, filters are adaptive as their parameters depend on the particular image
to which they are applied. In view of the implementation issues, the filters
are designed and work in the domain of orthogonal transforms that can be
computed with fast transform algorithms. The design of such filters is
discussed in Sect. 8.2. In Sect.8.3, this approach is extended to local adaptive
filters that work in transform domain of a window sliding over the image
and, in each window position, produce filtered value of the window central
pixel. Illustrative examples are presented that demonstrate edge preserving
noise suppression capability of local adaptive filters. In particular, it is
shown that such filters can be efficiently used for filtering signal dependent
noise, such as speckle noise. Sect. 8.4 extends this approach further to the
design of optimal linear filters for multi component images such as color,
multi spectral or multi modality images.
In real application, it very frequently happens . Very that noise randomly
replaces signal values that in this case are completely lost. In such situation,
the model of impulse noise is used. Filtering impulse noise, in general,
requires nonlinear filtering. Efficient and fast impulse noise filtering
algorithms that use simple linear filter and point wise threshold detector of
pixels distorted by impulse noise are described in Sect. 8.5. Other filters for
filtering impulse noise are described in Ch. 12.
All above mentioned filtering methods are aimed at correcting signal
distortions attributed to linear filtering and stochastic transformation units of
the imaging system model. Sect. 8.6 deals with processing methods for
correcting image and hologram gray scale distortions in the point-wise
nonlinear transformation unit.
In a broad sense, image restoration can be treated as applying to the
distorted signal a transformation inverse to that that caused the distortions
provided that the inverse transformation is appropriately modified to allow
for random interferences and other distortions that are always present in the
signals. This is how image reconstruction in holography and tomography is
usually carried out. The modification of the inverse transformation to allow
for random interferences and distortions is implemented as pre-processing of
holograms or, correspondingly, image projections before applying the
8 Chapter 1

inverse transfonnation and post-proeessing after the reeonstruetion

transfonnation. This teehnology is illustrated in Seet. 8.7.
As a rule, in image proeessing applieations, end user needs, for solving
his particular tasks, apply to images some additional proeessing to ease
visual image analysis. Such a processing is usually ealled image
enhaneement. Image enhancement methods are reviewed and illustrated in
Seet. 8.8.
In digital holography and digital image processing real images and
holograms are represented in computers as arrays of their sampies obtained
with one or another sampling deviee in its eertain setting. Meanwhile it is
very frequently neeessary to resample images to obtain, from available array
of sampies, sampies loeated in position other then those of available
sampies. The resampling assumes interpolation between available data.
Many interpolation methods for sampled data are known. Among them,
diserete sine-interpolation that satisfies diserete sampling theorem deseribed
in Ch. 4, has eertain advantages. Ch. 9 foeuses on properties and efficient
eomputational algorithms for discrete sine-interpolation and on using it for
image zooming, rotation, differentiating, integrating, polar-to-Cartesian
eoordinate eonversion and tomographie reeonstruetion.
A very important applied problem of image proeessing is image parameter
estimation, and, in partieular, loealization of objeets in images. Methods for
solving this problem are diseussed in Chs. 10 and 11.
In eh. 10, abasie mathematieal model of observation with additive sensor
noise is fonnulated and applied to the design of optimal deviee for
loealization of a target object with the highest possible aeeuracy and
reliability. Analytieal fonnulas are also obtained that eharacterize potential
aeeuraey and reliability oftarget loealization for single and multi-eomponent
images and non-eorrelated and eorrelated sensor noise.
Ch. 11 foeuses on the problem of reliable loealization of a target objeet in
clutter images when the localization is hindered by the presence in images
non-target background objects. The main issue in solving this problem is
how to reliably discriminate target and non-target objects. In the chapter, a
localization device is considered that consists of a linear filter and a unit that
finds coordinates of the signal highest maximum at the filter output. The
linear filter is optimized so as to secure the highest ratio of the filter response
to the target object in its location to standard deviation of the filter response
to background non-target image component. This problem is solved for
exact1y known target objects and for objects that are known to the accuracy
of their certain parameters such as, for instanee, scale and rotation angle, for
spatially homogeneous and inhomogeneous images. The solution ealled
optimal adaptive correlator is extended to the case of localization in color
and multi component images. Some practical implication of this solution and
its implementation in nonlinear optical and opto-electronic eorrelators are
also discussed and illustrated.
1. Introduction 9

An important place in the arsenal of methods for image denoising,

enhancement and segmentation belong to nonlinear filters. Quite a number
of nonlinear image processing filters have been reported in the literature. In
the Ch. 12, these filters are c1assified and represented in a unified way in
terms of fundamental notions of pixel neighborhood and of a standardized
set of estimation operations. The chapter also provides and illustrates
practical examples of several unconventional filters for image denoising,
enhancement, edge detection and segmentation. In conc1usion, possible
implementation of the filters in neuro-morphic parallel networks are briefly
discussed.
The last, 13-th chapter is devoted to the problem specific for digital
holography proper, the digital-to-analog conversion problem of encoding
computer generated holograms for recording on physieal optical media and
to analysis of distortions in reconstructed images associated with encoding
methods. As computer generated holograms and optieal elements are still
recorded using means that are not directly intended for this purpose, there is
no unique encoding method. In the chapter, the most simple and known
methods are described and analyzed.
In all chapters, exposition is extensively supported by diagrams,
graphie al and picture illustrations as the author shares the weIl known
Chinese saying that a pieture costs more than a thousand words.
 10 Chapter 1

REFERENCES

1. D. Gabor, Microscopy by reconstructed waveforms, 1, Proc. Royal Society, A197, 454-

487, (1949)
2. http://www.nobel.se/physics/laureates/1971/gabor-lecture.html
3. E. N. Leith, l Upatnieks, New techniques in wavefront reconstruction, JOSA, 51, 1469-
1473, (1961)
4. Yu. N. Denisyuk, Photographic reconstruction ofthe optical properties ofan object in its
own scattered radiation field, Dokl. Akad Nauk SSSR, 144,1275-1279 (1962)
5. T. Huang, "Digital Holography", Proc. ofIEEE, 59,1335-1346 (1971)
6. B. R. Brown, A. Lohmann, Complex spatial filtering woth binary masks, Appl. Optics, 5,
No. 6, 967-969 (1966)
7. T. S. Huang, B. Prasada, Considerations on the generation and processing of holograms
by digital computers, MlT/RLE Quar. Prog. Rep. 81, Apr. 15, pp. 199-205 (1966)
8. A. W. Lohmann, D. Paris, Binary Fraunhofer holograms generated by computer, Appl.
Optics, 6, No. 10, 1739-1748 (1967)th. Comp., v. 19, 1965, pp. 297-301
9. l W. Goodman, R. W. Lawrence, Digital image formation from electronically detected
holograms, Appl. Phys. Lett., 11, pp. 77-79, Aug. 1 (1967)
10. L. B. Lesern, P. M. Hirsch, J. A. Jordan, Kinoform, IBM Joum. Res. Dev., 13, 150,1969
11. M . . A. Kronrod, N. S. Merzlyakov, L. P. Yaroslavsky, Computer Synthesis of
Transparency Holograms, Soviet Physics-Technical Physics, v. 13, 1972, p. 414 - 418.
12. M.A. Kronrod, N.S. Merzlyakov, L.P. Yaroslavsky, Reconstruction of a Hologram with
a Computer, Soviet Physics-Technical Physics, v. 17, no. 2,1972, p. 419 - 420.
13. L. P. Yaroslavskii, N.S. Merzlyakov, Methods of Digital Holography, Consultans
Bureau, New York, London (1980) (Russian Edition: L. Yaroslavsky, N. Merzlyakov,
Metody tsifrovoy golographii, M., Nauka, Moscow, 1977)
14. Wai-Hon Lee, Computer Generated Holograms: Techniques and Applications, In: E.
Wolf, Ed., Progress in Optics, v. XVI, North Holland, pp. 121-230, (1978)
15. Dallas W. l, Digital Holography, in: Frieden, Ed., Computers in Optical Research,
Topics in Applied Physics, v. 41, Springer, Berlin, pp.29 1-366 (1980).
16. O. Bryngdahl, F. Wyrowski, Digital Holography - Computer -generated Holograms, in:
E. Wolf, Progress in Optics, XXVIII, Elsevier Science Publishers B. V., pp. 3-86 (1990)
17. D. Gabor, Theory of communication, Journ. Inst. Electr. Eng., vol. 93, 429, (1946)
18. D. Gabor, Communication Theory and Physics, Phi!. Mag., vo!. 41, No. 7, p.1161 (1950)
19. D. Gabor, Light and information, in: Progress in Optics, 1, ed. By E. Wolf, Amsterdam,
pp. 109-153, (1961)
20. F. T. S. Yu, Optics and Information Theory, Robert E. Krieger Pub!. Co, Malabar,
Flodira, (1989).
21. l W. Cooley, l M. Tukey, An Aigorithm for Machine Calculation of Complex Fourier
Series, Math.Comp.,v. 19,1965, pp. 297-301
 Chapter 2
OPTICAL SIGNALS AND TRANSFORMS

2.1 MATHEMATICAL MODELS OF OPTICAL

SIGNALS

2.1.1 Primary definitions and classification

Signals and signal processing methods are treated mathematically through

mathematical models. The very basic model is that of a mathematical
function. For instance, optical signals are regarded as functions that specify
relationship between physical parameters of wave fields such as intensity,
and phase and parameters of the physical space such as spatial coordinates
and/or of time. Fig. 2.1.1 provides c1assification of signals as mathematical
functions and associated terminology.
Examples of scalar optical signals are monochrome gray scale images.
Examples of vectorial (multi-component) signals are two component signals
that represent, for instance, orthogonal components of polarized
electromagnetic wave fields or leftlright stereoscopic images, three
component color images represented by their red, green and blue
components and multi spectral images that may contain more than three
components. Examples of I-D signals are audio signals or physical
measurements as functions of time. Images are modeled as 2-D functions of
co-ordinates in image plane while video can be regarded as 3-D functions of
image plane eo-ordinates and time. Volumetrie data are funetions of three
12 Chapter 2

Signals as matbemattcaJ functions

I

Scalar
------
Function value

Vectorial
-------
One-
Function arguments

MuJti-
(single (multi- dimensional dimensional
component) component) (l-D (2-D, 3-D,
signals signals signals) 4-D, etc.

--
signals)

/
Continuous
in
Continuous
in
~

-- Quantized
in values ""
Discrete in
arguments
vaIues arguments (quantized) (Discrete)
signals signals

Continoous signals
Signal
digitization •..1
~ /
....
Signal
.-.
I
Digital signals
I
reconstruction

Figure 2-1. Classification ofsignals as mathematical functions

space co-ordinates. Continuous optical spectrum color images can also be

treated as 3-D functions ofwavelength and co-ordinates in the image plane.
Discrete signals are signals specified as a list (sequence) of values.
Individual entries of this list are called signal elements or signal sampies.
Quantized signals are those that take values from a finite set of possible
values (quantization levels). Optical signals that carry information on optical
properties of real physical objects are continuous signals. They are modeled
as continuous functions both in terms of their values and their arguments.
Continuous signals are frequently called analog signals for they are
regarded as analogs of the corresponding physical objects.
At the other end of the classification we find digital signals, those that
one deals with in computers. In general, each digital signal can be regarded
as a single number. In practice, digital signals are represented as a sequence
of quantized numbers that may be considered as digits of this single number
and are called digital signal sampies. In this sense digital signals can be
regarded as being botb discrete and quantized. For digital signal processing
 2. Optical signals and trans/orms 13

one should convert analog signals into the corresponding digital ones. We
will refer to this conversion as signal digitization. Signal digitization always,
explicitly or implicitly, assumes that inverse conversion of the digital signal
into the corresponding analog signal is possible. We will refer to this digital-
to-analog conversion as signal reconstruction.
Throughout the book, we designate continuous signals by lower case
roman letters a,b, etc., and their arguments by x. Multi-component signals
will be designated by vectors of their components a = {ac J, c =1,2,••., C . For
multidimensional continuous signals, argument x is a vector variable with
components x = {x d J,d =1,2,••• ,D, where Dis the signal dimensionality.

2.1.2 Signal space

In the signal theory it is very convenient to give signals a geometrie

interpretation by regarding them as points in some functional space - the
signal space . In such a treatment, each signal is represented by a point in the
signal space and each point of the signal space corresponds to a certain
signal (Fig. 2.1.2). Signal space of continuous signals is a continuum.
Using the notion of the signal space, one can easily portray a general the
relationship between continuous and digital signals. Let us split signal space
into a set of sub-spaces, or cells that cover the entire space and assign, to
each cell, its index number. As a result, each index number will correspond
to all signals within the corresponding cell and, for each signal, one can find
the index number ofthe cell to which the signal belongs.
Suppose now that all cells are formed in such a way that all signals
within one cell can be regarded identical or indistinguishable for a given
application. We will call such cells equivalence cells. In each equivalence
cell, one can select a signal that will serve as a representative signal of all
signals within this cello In such a treatment, conversion of a continuous
signal into a corresponding digital signal (analog-to-digital conversion) is
assigning to the continuous signal the index number of the equivalence cell
to which it belongs. We will call this process general digitization. Inverse
process of converting digital signal into analog one (signal reconstruction) is
then finding, for each index number (digital signal), a representative signal
of the corresponding equivalence cello In practice, partition of the signal
space into a set of equivalence cells is "hard-wired" in analog-to-digital
converters and assigning representative signal to a number is "hard-wired" in
digital-to-analog converters.
The design of the partition of signal space into a set of equivalence cells is
only one of many signal processing tasks that require describing
mathematically distinctions between signals. To this goal, the concept of
space metries is introduced which treats distinctions between signals as a
distance between the corresponding points in the space.
14 Chapter 2

Representative
signal

Figure 2-2. Signal space, equivalency cells and representative signals

The signal spaee metries associates with each pair of signals in the space,
say, a(l)and a(2), a nonnegative real number d(a(l) ,a(2»). The method for
obtaining this number normally has the following properties:

(2.1.1)

(2.1.2)

(2.1.3)
 2. Optical signals and trans/orms 15

The meaning of the first condition is obvious. The second condition is

equally natural, although there are metrics for which this condition does not
hold. The idea behind the third condition, referred to as the "triangle rule",
formally expresses the following natural requirement of a metrics: if two
points are close, in the metrics, to a third point, they must be close to each
other. Examples of the metric most commonly used in the theory of signals
are presented in the Table 2.1 (second column)

T.able 2-J. Tl}'p}C

. al sIgna
. S ace metncs
Signal Design Deterministic definition Statistical definition
type ation AVis a statistical
averaging operator
Discrete LN ~Ja11) - a12)1 AVia!l) - a!2)IJ
signals
k=O
L2
N
N-l
Ila!l) - a!2)1 2 AV~ail) -ai2f)
k=O
AV~ail) - ai2f )
N-l
LNP
Ilail) -ai2
k=O
f
LMhN N-l 2 -
I Wk2lail) - a!2)1 (
k=O
{Wk2}are scalar weight
coefficients)
maxla(l) a(2)1 -
MN
k k- k
Continu Lx Jlal{x)-a2{x~ AV~al (x)-a 2 {x)}
ous x
signals L2
x Jla l{x)-a 2{xf dx Av~al{x)-a2{Xr )
x
LPx Jlal(x)-a1(xf dx AV~al(x)-a2(Xr)
x
Mx sup~al (x)- a2(x )), -
x
where sup {a{x )) -
minimal value that is not
exceeded by a{x)

In many signal and image processing applications, distinctions between

signals have to be evaluated statistically, or on average over a set, or
ensemble, of signals. This is the case, for instance, when one needs to
 16 Chapter 2

evaluate performance of a signal processing device or of an algorithm to be

used for a variety of signals from a certain dass or from a certain data base.
The relevant performance measure should characterize the performance for
the whole dass of signals. Therefore, one or another method of averaging
distance measure evaluated for every particular signal of the dass is required
as it is shown in Table 2.1 (right column).
In order to specify the way of statistical averaging one should introduce
characteristics of signal ensemble that are conventionally referred to as
signal statistieal eharaeteristies. The primary statistical characteristic is
probability of signals in the signal space. It defines a relative contribution of
signals into the averaging. For continuous signals, probability measure is
defined as probability density. For determination of signal space statistical
characteristics, mathematical models of signals as random processes are
used. For the theory of random signals readers are referred to dedicated
textbooks and monographs such as Refs. [1-4].
From the table one can see that any distance measure involves two items:
Loss function (we will denote it as LOSS(.,.)), that evaluates
numerically the distinctions between the individual signal
sampies (for discrete signals) or their values for particular values
of its arguments (for continuous signals). For instance, for LN
metric, loss functions is defined as an absolute value of the
difference between signal sampies.
Method for the averaging loss function values (for instance, by
summation over the signal extent or by statistical averaging over
a signal ensemble as it is shown in the table).
In image processing it is frequently more adequate to average the loss
function over both area of images from an image set and over all images in
the set. It is also useful sometimes to weight loss function values with
weights that depend on the signal arguments (an example of such weighting
one can see in Table 2.1 in the definition of metric LMh N ) or other signal
parameters. This leads to a family of "local" metrics (Ioeal eriteria) which,
for discrete signals, are defined as

(2.1.4)

where A V {.} designates the averaging over image data set. Local criteria
assume evaluation of signal distinctions "locally" around any particular point
k of argument of signal' s function. Applications of local criteria to the
design and substantiating adaptive linear and nonlinear filters will be given
in Chapts. 8 and 12 .
 2. Optical signals and trans/arms 17

2.1.3 Linear signal space, basis functions and signal representation

as expansion over a set of basis functions

An important feature of physical signals such as optical ones is that they

can be added, subtracted and amplified or attenuated. Mathematically, this is
modeled by the property of linearity of the signal space. The signal space is
called linear if:
1. An operation of signal summation (+) is defined such that, for
any two signals a(l) and a(2), a third signal a(3) = a(l) + a(2) can
be uniquely found.
2. The summation operation obeys the commutative and
associative laws:
a(l) + a(2) = a(2) + a(l)
a(l) + a(2) + a(3) = (a(l) + a(2))+ a(3) = a(l) + (a(2) + a(3))
3. There exists a "zero-signal" 0 such that a + 0 = a for all
signals of the space.
4. Each element a of the space may be put into the
correspondence with a unique opposite signal (- a) such that
a + (-a) = a - a =0 .
5. An operation 0
ofmultiplying signal by a scalar number is
defined such that, for any number a and any signal a , a signal
a . a can be uniquely found.
6. The following properties hold for the multiplication operation:
utuZa =u t (uza )=a z (uta);
(u l +uJa=u l a+u 2a; a(a l +a 2 )=aa l +ua 2 ;
l·a=a; 0·a=0;
Usually operations of signal addition and signal multiplication by scalars
are treated as the customary operations of point-wise (element-wise)
arithmetic summation and multiplication of signal values and the "zero-
signal" is considered to be a signal whose values are equal to zero for all its
argument values.
In linear signal space, one can specify a set of N signals and use them to
generate an arbitrary large set of other signals as a linear combination
(weighted sum)

(2.1.5)

of the selected signals that are called basis signals (basis funetions).
Obviously, the set of basis functions should be selected from functions that
can't be represented as a linear combination ofthe rest of the functions from
18 Chapter2

the set, otherwise the set of basis functions will be redundant. The signal
space formed of signals generated as a linear combination (2.1.5) of N
linearly-independent basis functions is referred to as an N-dimensional
linear space.
Each signal a in N-dimensional signal space corresponds to a unique
linear combination of basis functions {cP k } , Le. is defined by a unique set of
scalar coefficients ~ k }. An (ordered) set of scalar coefficients of
decomposition of a given signalover the given basis is the signal's
representation with respect to this basis.
The "zero-signal" may be treated as a reference signal of the signal space.
The distinction of signal a from the "zero-signal" displays the individual
characteristic of this signal. Mathematically, this characteristic is described
through the concept of the signal' s norm Ilall :

Ilall =d(a,O) (2.1.6)

The geometrical interpretation of the vector norm is its length.

Signal representation as an expansion over a set of basis functions (2.1.5)
is meaningful only when for each signal a, its representation ~ k} for the
given basis <P k can be found. To this goal, the concept of the scalar product
(inner product, dot product) (al ,al) of two signals is introduced as a
method for computing a (generally, complex) number that satisfies the
following conditions:

(a,a)~O; (a,a)=O, iffa=O;

(2.1.7)

where asterisk (*) represents a complex conjugate. Usually the scalar

product of two signals is calculated as an integral over their product:

(a.,a z )= fa. (x)a; (x}dx (2.1.8)

x
 2. Optical signals and transforms 19

This computation method satisfies the above conditions and, as we will see,
is a mathematical model of natural signal transformations such as those in
optical systems and signal sensors.
The concepts of signal scalar product and norm may be related by
defining the norm through the scalar product :

JJaJJ =(a,a). (2.1.9)

Those functions whose scalar product is zero are called mutually

orthogonal functions. If functions are mutually orthogonal, then they are
linearly-independent. Therefore mutually orthogonal functions may be used
as bases of linear signal spaces.
Having defined a scalar product, one can use it to establish a simple
correspondence between signals and their representation for the given basis.
Let ~ k} be the basis fimctions and {4> I} be the reciprocal set of fimctions
orthogonal to {cp k} and normalized such that

k:t;l
(2.1.10)
k=l

(symbol Ö k,l is called the Kronecker delta). Then signal representation

coefficients ~ k} over basis {cP k} can be found as

N-I N-I

Cl k = (a,cjlk) = LClI(cp"cjlJ=LClIOk,l' (2.1.11)

1;0 1;0

If basis ~ k} is composed of mutually orthogonal functions of unit norm:

(2.1.12)

it is reciprocal to itself and is called orthonormal basis. In such a basis

signals are representable as

(2.1.13)

where

(a,CPk)=Cl k = fa(x}p;(x}1x. (2.1.14)

x
20 Chapter 2

Knowing the signal representation over the orthonormal basis, one can easily
ca1culate signal scalar product

andnonn

(2.1.16)

Representations tx i ,{ß ~t)} and , tx i1)} ,{ß ~l)} of signals a and b over
t )}

two orthononnal bases ~ it )} and ~ !l)} are related by means of the

Parseval 's relationship

(2.1.17)

Up to now, it was assumed that signals are continuous. For discrete

signals defined by their elements (sampies), a convenient mathematical
model is that of vectors.
Let {a k} is a set of sampies of a discrete signal, k = 0,1,•.•, N -1 . They
define a vector-column a = {a k }. One can select a set of linearly
independent vectors {<Pr} = {~r,k}}:

(2.1.18)

for arbitrary coefficients {A. r '# o} as a basis for representing any signal
vector:

N-t

aN = LCIr<Pr . (2.1.19)
r~O

An ordered set of basis vectors can be regarded as a matrix

(2.1.20)
2. Optical signals and transforms 21

and signal expansion of Eq. 2.1.18 as a matrix product of the vector-column

of coefficients a = {a k} by the basis matrix:
(2.1.21)

In signal processing jargon, matrix TN is called transform matrix and signal

representation defined by Eq. 2.1.21 is called signal transform. If the
transform matrix TN has its inverse matrix T;l such that

(2.1.22)

where I is the identity matrix:

IN = {ö k-'}, (2.1.23)

vector of signal representation coefficients can be found as

(2.1.24)

Such matrix treatment of discrete signals assumes, that signal scalar product
is defined as a matrix product

(2.1.25)

Transforms T N and T;' are called mutually orthogonal and signal vector
transformations described by Eqs. (2.1.20) and (2.1.23) are called direct and
inverse transforms, respectively. Transform matrix that is orthogonal to its
transpose

(2.1.26)

is called orthogonal matrix. Transform matrix TN that is orthogonal to its

complex conjugate and transposed copy (T~ )" :
TN . (T*N )" =I N (2.1.27)

is called self conjugate matrix and the transform is called the unitary
transform. For unitary transforms, the Parceval's relationship holds:

(2.1.28)
 22 Chapter 2

2.1.4 Integral representation of signals

The discrete representation of signals

(2.1.29)

described in Sect. 2.1.3 can be extended to a continuous representation

obtained were we to substitute in Eqs. 2.1.5 the basis function index k , by a
continuous variable, say / , and replace sum by an integral:

a(x)= fa(f}p(x,f)df (2.1.30)

It is quite natural to extend this approach to the definition of a(J) by

introducing reciprocal functions <I> (J,x), to obtain an analog of Eq. 2.1.14:

(2.1.31)

The function a{f) is called the integral transform of signal a{x), or its
spectrum, over continuous basis {<I> (f, x)} called transform kerne!. The
reciprocity condition for functions <p (x, f) and <I> (f, x) or the condition of
transform invertibility may be obtained by substituting (2.1.31) into (2.1.30):

(2.1.32)

where

o(x,~)= f<p(x,f~(J,~)df. (2.1.33)

F

The function Ö (x,~), defined by Eq.(2.1.33), is called the delta-function

(Dirac delta). It is a continuous analog of Kronecker delta (Eq. 2.1.1 0) and
is a symbol of orthogonality of kerneis <p (x, /) and <I> ( / , x), or, in other
words, of invertibility of the integral transform of Eq. (2.1.30).
Formula (2.1.32) may be regarded as a special case of the integral
representation with the de1ta-function as its kerne!. One can also treat
2. Optical signals and transforms 23

discrete signal representation (Eq. 2.1.13) as a special case of its integral

representation:

a(/}= ja(x)p(/,x)dx= j[~akCPk(x}}(/,x)dx= ~akO(/'/k}

(2.1.34)

where

8 (/,/,.) = JCPk(X)P(/,x)dx. (2.1.35)

x

Using Eq.(2.1.33) as an analog to Eq.(2.1.12), one can generalize the

concept of the orthogonal basis as well.

Jcp (x, /}p . (x, /)// = 0 (x,~ ) (2.1.36)

F

A basis satisfying this relationship is called selJ-conjugate. For self-

conjugate bases, signal scalar product is invariant to the bases

ja 1(x)a;(x)dx = ja 1(x)x;(/)d/. (2.1.37)

x F

Eq. (2.1.37) is a continuous analog to the Parseval's relation (2.1.17). In

particular,

Jla (xt dx = J~l (xt df·

x
1
F
(2.1.38)
 2.2 SIGNAL TRANSFORMATIONS

In optical systems, optical signals undergo various transformations. In

general, one can mathematically treat signal transformations as a mapping in
the signal space. For digital signals, this mapping, in principle, can be
specified in a form of a look-up table. However even for digital signals this
form of the specification is not practically feasible owing to the enormous
volume of the digital signal space (a rough evaluation of it one can see in
Sect.3.l). For discrete and continuous signals such a specification is all the
more impossible.
Therefore in practice, signal transformations are mathematically
described as a combination of certain "elementary" transformations, each of
which is specified with the help of a relatively small sub set of all feasible
input-output pairs. The most important of these are point-wise
transformations and linear transformations.
Signal transformation PWT(.) is called point-wise if it results in a
signal whose value in each point of its argument (coordinate) depends only
on the value of the transformed signal in the same point:
b(x =l;)= PWT[a(x =l;)]. (2.2.1)
For digital signals, point-wise transformations are applied to individual
sampies of the signal and are specified by a look-up table with number of
entries equal to the number of signal quantization levels. For discrete and
continuous signals, it is assumed that point-wise transformations are
specified in a form of mathematical functions of a single argument. This
function is referred to as system transfer function. If the transfer function
does not change with signal coordinates, the transformation is called
homogeneous transformation. Otherwise the transformation is
inhomogeneous transformation.
A typical example of the element-wise optical transformation is the
relation goveming the conversion of light radiation into the degree of
darkening of a photographic plate. Units responsible for element-wise signal
conversion are widely used in models of optical and photographic systems.
Transformation L(.) is linear if it satisfies the superposition principle:

(2.2.2)

Obviously, linear signal transformations are defined only for linear signal
space and linearly transformed signals also form a linear signal space.
Therefore for linear transforms, one may introduce the notion of the sum of
2. Optical signals and trans/orms 25

transfonns, i.e. L Ln' as weIl as the product of transfonns, fI Ln' The

n n
physical equivalent of the product is aseries (cascade) connection of units
realizing the transfonn-factors. The physical equivalent of the sum of linear
transfonns is a parallel connection of units realizing the transfonn-items.
Thanks to the linearity of the transfonns, their multiplication is distributive
with respect to addition:
LI (L 2 + L3 ) = LIL2 + LIL3 ;
(2.2.3)
(LI + L2 )L 3 = LIL3 + L2 L3
If a linear transfonnation L perfonns a one-to-one mapping of a signal,
then there exists an inverse transfonnation L- I such that

(2.2.4)

A linear transfonnation of discrete signals is fully characterized by a matrix

A = {Ak,n} linking the input a = ~ k} and output b = {ß k} pairs of the
linear transfonnation b = La , respectively:

(2.2.5)

Thus, in order to describe a linear transfonnation of a sequence of N

numbers, it suffices to specify a matrix of N 2 numbers. Equation (2.2.5)
evidences the fact that in the general case, linear transfonns are not point-
wise, and that they become so only if transfonnation matrix is a diagonal one
({Ak,n = Ak8 k-n h·
For continuous signals, integral transfonns are the instruments by which
one produces a meaningful description of linear transfonnations. Let a
continuous signal a(x) is specified by means of its spectrum a.(f) on
bases {CP(x,f~~(f,x)} (Eq. 2.1.30, 31 and, in a special case of delta-
function base, Eq. 2.1.32). Then the result b(~) of applying to this signal a
linear transfonnation L

b(~) = La (x) = L fa.(f}P(x,f)Jf = fa.(fXL<p(x,f)]df (2.2.6)

F F

is fully specified by the response h(~,f)= L(<p(x,f»of the linear

transfonnation to the transfonn kerne I
26 Chapter 2

b(~)= fa(/)h(~,I)dI. (2.2.7)

F

For the representation of linear transformation output signal b(~) also

via its spectrum ß(I) over basis ~ (/,~ )

ß(/)= fb(~~(/,~~, (2.2.8)

speetra ß(I) and a (I) of a signal and its linear transformation are also
related with an integral transform

ß(/)= fa(p)H(/,p)dp (2.2.9)

F

whose kernel Ht/, p) in its turn is an integral transform, with kernel

~ t/,~), ofthe linear transformation response ht~, I) to basis <p tx, I) in
which spectrum of the signal is specified:

(2.2.10)

Signal integral representation (2.1.32) via delta-funetion is of espeeial

importance for the characterization of linear transformations. For such a
representation, the relationship between output signal b(x) and input signal
a(x) of a linear transformation (Eq. 2.2.7) is reduced to
b(x)= fa(~ }h(x,~ ~ , (2.2.11)

where h(x,~) = L(ö (x,~)) is linear transformation response to a delta-

function called impulse response or point spread function of a linear system
that implements this transformation.
Cascade combination of units performing linear and point-wise signal
transformations (Fig. 2.3) is the basie model used to deseribe mathematieally
imaging and holographie systems.

1-- - - - - - - - - - - - - - - - - - - - - - -I

....
I ..

Input 1
Linear
transformation
...... Point-wise nonlinear
transformation
+-.
IOutput
Imaging system
~------------------------
Figure 2-3. Basie model of imaging and holographie systems
 2. Optical signals and trans/orms 27

2.3 IMAGING SYSTEMS AND INTEGRAL

TRANSFORMS

2.3.1 Direct imaging: Convolution integral

Imaging systems are systems that create images of real or phantom

objects. In describing imaging systems it is always assumed that there exist a
physical or imaginary object plane and an image plane and that the primary
goal of imaging systems is reproducing, in image plane, distribution of light
reflectance or whatever other physical property of an object in the object
plane.
Apparently the very first imaging devices were painters (Fig. 2-4). The
first artificial imaging device, was, seemingly, camera-obscura (pinhole
camera, Latin for 'dark room')) that dates back Aristotle (384-322 BCE) and
Euclid (365-265 BCE) (Fig. 2-5 ). Camera obscura was the ancestor of the
modern photographic and TV cameras (Fig. 2-6).

Figure 2-4. A woodcut by Albrecht Dürer showing the relationship between the light
distribution on an object and image plane (adopted form R. Bracewell, Two-dimensional
imaging, Prentice Hall Int. 1995)
28 Chapter2

Objective lens

----
------ Image
b(x. v)
------ --- ---

Objectplane (~,11)

Figure 2-6. Schematic diagram of optics of photographie and TV cameras

Many other modem imaging devices such as X-ray, gamma cameras,

scanning electron, acoustic, scanned proximity probe (atomic force, tunnel)
microscopes work on the same principle of imaging one point in object plane
into a corresponding point in image plane. Being, in their optical part,
devices that satisfy the superposition principle, they are most naturally
characterized by their point spread function h(x, y; ~ , Tl ) :

b(x,y)= JJa(I;,Tl)h(x,y;I;,Tl~dll , (2.3.1)

xv

where integration is assumed to be carried out over the object plane. Integral
ofEq. (2.3.1) implicitly shows image ofa point source located in coordinates
(~, Tl)·
In general, the point spread function is space variant: image of a point
source changes depending on its location in object plane. An important
special case is that of space invariant imaging systems in which image of a
point source is invariant to the point source location. For space invariant
systems, point spread function is a function of difference of coordinates in
object and image planes:

(2.3.2)

Imaging equation (2.3.1) for space-invariant systems takes form:

b(x,y)= JJa(~,Tl}h(x-~,y-Tl)d;dTJ.
'" '"
(2.3.3, a)
2. Optical signals and transforms 29

called 2-D convolution integral. Correspondingly, I-D convolution integral

is defined as
00

b{x)= fa{~}h{x -~~ (2.3.3, b)

-00

Note that integration limits in Eqs. (2.3 .3) are infinite. This is a direct
consequence of the assumption of space invariance.
Point spread function is the basic specification characteristic of imaging
systems. It is used for certification of imaging systems and characterizes the
system capability to resolve objects called imaging system resolving power.
Given noise level in the signal sensor, it detennines variance of error in
measuring coordinates of a point source ([5], see also Sect. 10.3). In
particular, one can show that, for separable PSF h{x,y)= hx{x}hy{Y),
variances cr; and cr: of the error in measuring (x,y) coordinates of a point
source are inversely proportional to the energy of the corresponding
derivatives of PSF:

(2.3.4)

On a qualitative level it is commonly accepted that resolving power of

imaging systems is defined by the Rayleigh 's criterion: two point sources
are considered resolved if minimum between two corresponding PSF peaks
does not exceed 80% or so of the peak maxima (Fig. 2-7).

------------~ ....

Figure 2-7. Rayleigh's criterion ofthe resolving power

30 Chapter 2

Imaging systems described by the convolution integral (Eq. 2.3.3) are, in

general, not invertible: one can not perfectly restore system input image
a(x, y) from the output image a(x, y) . The problem of the restoration
belongs to so called inverse problems. Methods for image restoration will be
discussed in Ch. 8.
In conclusion note that the assumption of imaging system space
invariance is most frequently one used in the description and analysis of
imaging systems and image restoration. It leads to many simplifications both
in system analysis and in image processing. The advantages of space
invariance of system PSF can sometimes also be utilized even if the
assumption does not satisfactorily hold. This can be achieved if one converts
signals from original coordinate system into a in a transformed one in which
system PSF becomes space-invariant. After space invariant processing in the
transformed co ordinate system signal should then be converted back to the
original coordinates. This principle is illustrated in Fig.2-8 .
..............................•..••...........•.•..•.• ......... ~

..
~
Coordioate
cooversion ~
Spau
invanant
~
Inverse
coordinate ...
processlug cooversion

Space variant system

.......•.•.•.....•..•.•.........................................
Figure 2-8. Principle of space invariant representation of space variant systems

In digital processing this methods requires signal resampling. Methods for

discrete signal resampling will be discussed in Ch. 9.

2.3.2 Imaging in Fourier domain: Holography and diffraction

integrals

Invention ofholography by D. Gabor ([6,7]) was motivated by the desire

to improve resolving power of electron microscopes that was limited by the
fundamental limitations of the electron optics. The term "holography"
originates from Greece word "holos" (11001000'). By this, inventor of
holography intended to emphasize that in holography full information
regarding light wave, both amplitude and phase, is recorded by means of
interference of two beams, object and reference one. Holography remained
an "optical paradox" until the invention of lasers. The first implementations
of holography were demonstrated in 1961 by E. Leith and J. Upatnieks ([8])
 2. Optical signals and trans/orms 31

and by Yu. Denisyuk ([9]). The principle ofhologram recording is iIlustrated

in Figs. 2-9 and ofhologram reconstruction in Fig. 2-10.

Recording
Source of medium

Object beam
AObJ exp(i2n:<1> obi )

Reference beam
A~I exp{i2n:<1>,../ )

Figure 2-9. Schematic diagram of recording holograms

Radiation scattered by an object (object beam) interferes at a recording

medium with a coherent reference beam. The medium records the
interference pattern. Hologram is the result of the recording. Physical
recording media are sensitive to only energy of radiation. Therefore recorded
signal H(x, y) is proportional to the squared magnitude of the sum of
interfering beams:

(2.3.5)

where A Obi exp(i<l> Obi) and Are! exp(i<l> Te!) are complex amplitudes of object
and reference beams, respectively.
32 Chapter 2

For reconstruction of the hologram, one should illuminate the hologram

with a light beam of the same phase distribution as that of the reference
beam Ao exp(i<l> Tel)' The hologram (Fig. 2-10) modulates the reconstruction

Scattered

Source of coherent
light

, /

I ~' I
J, ,

Object bellm /f
;' I I"
",,/ I I ,"
ADbjA"IAo exp(i<l> Dbj) ;' /,/' / J'
" I , t
--------T:."'-iIII ,/ " ", ' /
'/
Reconstruction
beam
Ao exp{i<l>"1 )

Figure 2-10. Schematic diagram ofhologram playback

beam by amplitude and phase. As a result, three beams are generated: a

beam AObjA,efAO exp(i<l> obJ that has the same phase distribution as that of
the object wave and whose magnitude is proportional to that of the object
wave, a scattered reconstruction beam (A:bJ + A:ef )A o exp(i<l> ,ef)' and a
beam AobjA"IAo exp(i(2<l> ,ef - <l> obj )). The former is the beam that
reconstructs the object wave front (provided that A,ef and Ao are constant).
The latter reconstruct a "conjugate" imaginary object wave front.
The process of recording holograms is a good example of a two-unit
imaging system of Fig. 2.3. Point-wise nonlinear transformation unit in this
process describes recording interference pattern while linear transformation
unit refers to propagation of light from the object to the recording medium.
 2. Optical signals and transfarms 33

Consider 1-D model of free space wave propagation (Fig. 2-11). Let
complex amplitude of wave front of a monochromatic radiation in
coordinate x at the object ('input") plane is a(x). Let also hologram
("output") plane is situated at distance D from the object plane. At the
hologram ("output") plane at a point with coordinate J, this wave being
traveled a distance ~ D 2 + (x - f Ywill be accordingly attenuated and

obtain a phase delay 21t ~D 2 + (x - f Y/')..,

where ').. is the coherent
radiation wavelength. For the sake of simplicity, we will disregard the effect
of wave polarization on their superposition that is relevant for
electromagnetic radiation and is not for acoustic waves. Then one can
conclude that the point spread function of free space wave propagation is

Input plane Output plane

x --_
a{x )exp[ i2rr [ ~D2 +{x- fY
A II

Figure 2-11. Free space wave propagation point spread function

(2.3.6)

Obviously, 2-D point spread function offree space wave propagation from
input plane (x, y) to output plane Vx'
fy) is
34 Chapter2

(2.3.7)

The relationship between complex amplitudes a(x,y) and a{tx,ly) ,

respectively of the wave front in the input and output planes is then:

Integral ofEq. 2.3.8 is calledKirchhoffs integral ([10]).

Assume now that distance D between the input object plane (x,y) and
the observation output plane (tx' I y ) is substantially larger than maximal

distances ((x - Ix Y+ (y - I y Y t 2
involved in Kirchhoffs integral (2.3.8)
so as one can approximate

(2.3.9)

In this approximation, Kirchhofes integral is reduced, with omitting a sca1ar

multiplier, to its Fresne1 approximation:

a Ix' I y
( )
= -LIa(x, y)exp[(x
00 00

in
- Ix Y+ (y - I )2]dxdy .
fJ) y (2.3.10)

that describes the so called near zone diffraction. Eq. 2.3.10 can
altemative1y be written as
2. Optical signals and trans/orms 35

-LIa{x, y}exp(Xl
00 00 + l)
irc d
[ xl + yJ ]dxdy.
exp - i2rc " ')J) y (2.3.11)

If, further, distance D is so large that one can neglect phase variations

( fl Jl J (l l)
rc +
x ')J) Y and rc x ; ; ,we arrive at Jar zone diffraetion, or

Fraunhofer approximation ofKirchhoff's integral:

(2.3.12)

Eq. 2.3.12 describes also relationship between wave front complex

amplitudes in front and rear focal planes of lenses. Consider l-D model of a
lens (Fig. 2-12). Lens is fully defined by its property to convert light from a
point source in the lens focal plane into a parallel beam tilted according to
the position of the source with respect to lens optical axis. One can use this
property to determine point spread function of a lens.
One can see from the diagram in Fig. 2-12 that, at point I in the lens
rear focal plane, the plane wave front originated from a point source of
monochromatic light in coordinate (- x) for x«F (paraxial approximation),
is delayed with respect to point at the lens optical axis by

distance ~= I( -; ), where F is lens loeal length. From this one can

conclude that point spread function of optical systems whose input and
output planes are front and rear focal planes of a lens is

h{x, I) = exp{i2rc~ I A. ) = exp( - i2rc : ; ) (2.3.13)

and complex amplitudes a{x) and a{/} of monochromatic light in front

and rear focal planes of a lens are linked by a relationship:

a{f} = fa {x }exp(- i21t

x
fx)dx'
AF
(2.3.14)
36 Chapter2

----------------------
-------------
--------
------ I
-- -----------.1
I
•
I
I
_____________ F_______ ~---------F-----------
I

Figure 2-12. Schematic diagram ofwave propagation between focal planes of a lens

or, in 2-D denotation,

(2.3.15)

It is more convenient to rewrite Eqs. 2.3.8, 2.3.10, 2.3.12, 2.3.14 and 2.3.15
in dimensionless coordinates normalized, correspondingly, by .J').J) or by
.JAF . In such normalized coordinates, obtain, neglecting irrelevant constant
factors:

uK(Jx,fy )=

~~ eX+i'~d~ +d,[{X- f.)' +{v- fJ }

J Ja(x, y) dxdy (2.3.16)
-00 -00 ~dt + d [(x - fJ2 + (y - fy)j
Ä

where d Ä =DIA is a dimensionless distance parameter,

-00 -00

(2.3.17)
2. Optical signals and transfarms 37

and

(j) = fa(x )exp(i21tjx }ix ;

00

a F (2.3.18)
-00

00 00

aF Vx,fy )= f fa(x,y)exp[i21t Vxx + fyy)}adY. (2.3 .19)

-00 - 00

We will refer to Eq. 2.3.16 to as integral Kirchhof transform. Integral

transforrn defined by Eq. 2.3.17 is called integral Fresnel transform.
Integral transforrns defined by Eq. 2.3.18 and 19 are I-D and 2-D integral
Fourier transforms.
It follows from the above that optical system that consists of a lens and
input and output planes in the lens focal planes and that uses monochromatic
radiation acts as a Fourier transforrner, I-D with a cylindrical lens, and 2-D
with spherical one, of the wave front at the lens front focal plane. One can
also use two such Fourier transforrn systems in cascade and place into their
common focal plane a spatiallight modulator that is capable of modification
of the incident wave front magnitude or/and phase (Fig. 2-13). An optical
system buiIt in this way is a simplest coherent optical image processing
system. Image processing and target loeation methods that ean be
implemented in such systems are discussed in Chs. 8, 10 and 11 .

Parallel beam of
monochromatic SpatiaJ Jight
light modulator
,, ~" ------- ----
,,
,-
, ... ~
~ -'
~

" ... . . .
<___ F__
~
~
~

_ ......~ ___ .F____ ___ f ___ 2 __ ..E ___ _

Input I -SI Fourier Fourier 2-nd Fourier Output

image Jens plane lens image

Figure 2-13. Schematic diagram of an optical image processing system

38 Chapter 2

From schematic diagram in Fig. 2-11 it follows also that lens itself can be
considered as a spatial light modulator that adds to the phase of the incident
light a component that is a quadratic function of a distance from the lens
optical axis. Indeed, a spherical wave at point ~ in lens plain from a point
source at the optical axis in the input plane has complex amplitude

(2.3 .20)

In paraxial approximation, when size of the lens is much less than its focal
distance, wave amplitude is approximately

(2.3.21)

Because this spherical wave is converted by the lens into a plain wave,
complex transparency of the lens is then

(2.3.22 )

This property of lens allows to regard an optical system of three lenses

shown in Fig.2-14 as an optical Fresnel transformer, 1-0 with cylindrical
and 2-D with sphericallens.

Input plane Fourier plane Output plane

I I
I I
I I
I I
I I
I I

t
: I
-r --------------- - ---------------
I I
I I
I I
I I
I I
I I
I F F I

Figure 2-14. Schematic diagram of an optical Fresnel transformer

2. Optical signals and trans/orms 39

2.4 FOURIER TRANSFORM AND ITS

DERIVATIVES
Fourier transfonn plays an exceptional role in digital holography and
image processing. Apart of its fundamental role in mathematical
representation of signal transfonnations in wave propagation and image
fonnation described in the previous section, it has, in its discrete
representation as Discrete Fourier Transfonn (DFT, see Chapt. 4), a
remarkable property of efficient computation via Fast Fourier Transfonn
(see Chapt. 6). As we will see it later in this and next sections, integral
Fourier transfonn represents also a very convenient tool for linking all linear
transfonns. In this section, we discuss main properties of integral Fourier
transfonn and such its derivatives as Cosine Fourier Transfonn, Hankel
transfonn, Hartley transfonn and Mellin transfonn. We will describe here
also Fresnel, Fractional Fourier and Hilbert transfonns that are c10sely
associated with Fourier transfonn.
In order to make derivations more compact, we will, when it will be
convenient, treat two-dimensional coordinates (x,y) and (/x'/y) as two-

component vectors X = [ ;] and f = [ ; ,]. In such denotation, 2-D integral

Fourier transfonn is written as

a(f)= ]a(x)exP(i21tX"f)tx. (2.4.1)

where X" is transposed vector x. Eq. 2.4.1 can also be generally regarded
as a multi-dimensional integral transfonn assuming that X and f can be, in
general, multi component vectors.

2.4.1 Properties of integral Fourier Transform

The most important properties of integral Fourier transfonn are the

following.

Invertibilitv
Integral Fourier transfonn is invertible. According to the integral Fourier
theorem ([11]) signal a(x) can be found from its Fourier spectrum a(f) as

a(x)= ]a(f)exP(-i21tf" .x}it = ~~~ fa(f)exp(-i21tfI' .x}tr, (2.4.2)

-00 F
40 Chapter 2

where F is the area of integration and

a(x)= Ja(f)exP[-i2nf ,r .x}tt. (2.4.3)

is called inverse Fourier transform. Therefore, the kernel of the integral

Fourier transform is self-conjugate with delta function that symbolizes the
self- conjugacy defined as

O(x'~)=l~~ fexp[ilnftr .(x-~)}tt. (2.4.4)

F

In the special case of 1-0 integral Fourier transform:

-s )]},
F

ö (x,,) = ~~ fexp[i21tf(x - ,)]df = ~~{2F sine [21tF(x (2.4.5)

-F

where

. ()
SIne x =sinx
-- (2.4.6)
x

is so called sinc-function. Sinc-function can be regarded as Fourier

transform

sine(21tFx ) = } rect( f + F ) exp(i21tjX ~ (2.4.7)

-00 2F

of a rect-function:

1,0::;;x<1
reet ( x ) = { . (2.4.8)
0, otherwise

It is important to note that the invertibility of integral Fourier

Transform is a mathematical idealization. In practice, integration in Fourier
transform is always implemented in finite limits defined by aperture of
optical systems, dimensions of lenses and similar factors. These limitations
are referred to as bandwidth !imitations. Owing to the bandwidth
!imitations, artifacts known as Gibb 's effects may appear in form of
2. Optical signals and trans/orms 41

oscillations at signal sharp edges whenever Fourier transform is carried out.

More on Gibb's effects see in Sect. 3.2.2.1.

SeDarabi/itv. It follows from the definition of multi-dimensional integral

Fourier transform (Eq. 2.4.1) that it is separable to separate one-dimensional
transforms.

fexp{i21tX1h)ttt1 fexp{i21tX2h)ttt2 .... fa{x1,•••,XM)exp{i21tXMfM)ttt M

00 00 00

(2.4.9)

Shi(t theorem Absolute value of signal Fourier spectrum is invariant to

signal shift. Signal shift causes only linear modulation of the phase of the
spectrum.

Proo! From Eq. 2.4.3 it follows that

a(x-x o)= fa(f)exp[-i21tftr . (x-xo)Jn:=

(2.4.10)
ffx (f)exp(i21tf /' . X o)]exp[- i21tf /' . x}lr

which implies that if Fourier spectrum of a{x) is a{f), spectrum of

a{x - X o) is a{f)exp(i21tf tr • x o) and that absolute value of signal Fourier
spectrum is invariant to the signal shift.

Rotation theorem: Signal rotation results in the same rotation of its

spectrum.

Proo! In vector-matrix denotation of signal and spectrum coordinates,

signal rotation by an angle S can be represented as multiplication of
coordinate vector X by a rotation matrix Rots

-SiRS] . (2.4.11)
cosS
42 Chapter 2

From Eq. 2.4.3 obtain:

a(Rots .x)= fa(f)exp(-i21tf1r . (Rots .x)}tt=

-oe

fa (f)exp(- i21t (flr . Rots)· x}tt = fa (f)exp( - i21t ((Rots f ·f

-IX) -00
t .x )df =
fa(f)exp(-i21t(Rot-s .ff .X}tr.
-oe

(2.4.12)

Changing variables Rot_s . f ~ fand by virtue ofthe fact that the Jacobian
of this coordinate conversion is

J(Rot-s) = cos 1 So + sin 1 S =1 , (2.4.13)

obtain finally:

(2.4.14)

Therefore if Fourier spectrum of a(x) is a (f), spectrum of a{Rots . x) is

a{Rots .f).

Scaling theorem: Scaling signal coordinates S times causes I/S scaling ofits
spectrum magnitude and spectrum coordinates.
Proof

oe <Xl

a(SX)= Ja (f)exp[- i21tf· (SX)]af = Ja (f)exp[- i21t (Sf). x]af =

-oe -oe

(2.4.15)

Therefore, ifFourier spectrum of a(x) is a(f), Fourier spectrum of a(SX)

is ~a(~ f).
2. Optical signals and transforms 43

Convolution theorem: Inverse Fourier transfonn of a product a (f}ß (f) of

fa(~}b(x - ~~
00

spectra of two signals a(x) and b(X) is convolution of

these signals.
Proof

fa (f)ß (f)exp(- i21tf x}tr =

-00
tr

lI.(~)exP(i2xf'~~]ß (f)expG i2xf" X )Ir = (2.4.16)

_Ia(~tIß (f)exp(- i2nf (x -~)}tr ]~ = _Ia(~}b(x -~~

tr

Convolution theorem links Fourier transfonn and convolution integral.

As it was shown in Sect. 2.3.1, convolution integral describes image
fonnation in space invariant imaging systems. Convolution theorem shows
that image fonnation processes can also be treated in tenns of modulation of
image Fourier spectra by Fourier transfonn R(f) of convolution integral's
kerneI, impulse response h(X} of the space invariant system:
00

H(f}= fh(x}exp(i2nfx)dx. (2.4.17)

Function Rlf) is called systemfrequency response (the tenn customary in

signal processing and communication) or modulation transfer function
(MTF, the tenn customary within optical community).

Symmetry properties
Signal and their spectra symmetry properties are summarized in the
following equations in which symbol * means complex conjugation.
a(x)= ±a(- x)< FourierTransform )a (f)= ±a(- f); (2.4.18)

a(x) = ±a' (X )< FourierTransform ) a (f) = ±a ,(- f); (2.4.19)

a(x)= a(- X)= ±a'(x}< FourierTransform ) a(f}= a(- f) = ±a . (- f); (2.4.20)

a(x)= -a(- X)= ±a' (X )< FourierTransform )a(f)= -a (- f)= ±a' (- f); (2.4.21)
44 Chapter 2

2.4.2 Special cases of the integral Fourier transform: integral

Cosine, Hartley, Hankel and Mellin transforms

2.4.2.1 Integral Cosine trans form

When a(x) is an even function (a(x) = a(- x)), integral Fourier

Transform in reduced to integral eosine transform:

a(f)= fa(x)cos(21tXtrf)1x. (2.4.22)

Proof:

a (f)= }a(x )exp (i21tx tr f}lx = }a(- X )exp (i21tx tr f}lx =

-00 -00

_! a(x )exp (- i21tX tr f}lx = a(f)= a (f); a(f) =

1a(x )cos (21tX tr f}lx

(2.4.23)

Such a transformation is carried out in incoherent optieal information

proeessing and holographie systems ([12-15]).
By virtue of symmetry property of integral Fourier trans form
(Eq.2.4.15), inverse eosine transform is identical to the direet one:

(2.4.24)

Multi-dimensional eosine transform is not separable to I-D transforms. It

is separable if a(x) is even in all dimensions separately. In particular, 2-D
separable eosine transform

a Vx' fy)= fo fa(x, Y)cos(21tfxx )cos{21t/yY ~JY .

0
(2.4.25)
 2. Optical signals and transforms 45

Fig. 2-15 illustrates 2 types of symmetry that correspond to inseparable (left

image) and separable (right image) integral eosine transforms.

Figure 2-15. Two types of image even symmetry observed in images reconstructed from
computer generated Fourier holograms.

An important characteristic properties of integral eosine transform is

that it has areal valued kerne!.

2.4.2.2 Hartley transform

Yet another derivative of integral Fourier transform with real valued

kerne I is Hartley translorm ([16]). 1-0 Hartley transform is defined as

a(j)= Ja(x Xcos(27t.fx)+ sin(27t.fx)]dx .

00

(2.4.26)

With a multiplier fi , it is equivalent to

Ja(x )cos(27t.fx - 7t /4)dx .

00

a (j) = (2.4.27)

Similarly to the eosine transform, Hartley transform and its inverse are
identical. For real valued signals, the following relationships linking their
Fourier a F (I) and Hartley a H (j) spectra may be readily derived as
46 Chapter2

where Reta. J and Imta. Jare real and imaginary parts of a. .

As in the ease of the eosine transform, separable

and inseparable

(2.4.30)
-00-00

versions of 2-D Hartley transforms are possible.

2.4.2.3 Hankel transform

If a 2-D signal a(x,y) is cireularly symmetrie that is, in polar coordinate

system (r,S), (x = rcosS; y = rsinS ), it is a funetion of single variable
r:

a(x,y) = a(r) , (2.4.31)

2-D integral Fourier transform is redueed the so ealled Hankel transform:

fra(r ).10 (2xlr )dr ,

00

a. (I) = 2x (2.4.32)
o

where J o(.) is zero-order Bessel funetion of the first kind:

(2.4.33)
 2. Optical signals and transforms 47

One can show this by replacing Cartesian coordinates (x,y) and Vx,f) in
2-D Fourier transform by polar coordinates (r,3o) and (/,8), respectively:

-00-00

"""
f fra{r )exp~2n (lx r cos3o + f y r sin 30 )}txdy =
0-"
(2.4.34)
"" "
a{/,8)= f fa{r )exp[i2n (cos8 cos3o + sin8 sin30 )r/lrdrd3o =
o -7t
"" 7t ""

fa{r )rdr fexp{i2n cos{3o -8)}ie = 2n fa{r Yo{2nr/)rdr

o -7t 0

Integral Fourier transform a )/x,/y) and Hankel transform aHJ/) of a

radial symmetric signal a{r) (Eq. 2.4.29) are related as folIows:

(2.4.35)

From inverse 2-D Fourier transform one can derive that inverse Hankel
transform coincides with the direct one:

""
a{r)= 2n fa {/Yo{2n/r)/d/ . (2.4.36)
o

As one can see, Bessel function plays in Hankel transform role which
complex exponential function plays in Fourier transform. The role played by
sinc-function in Fourier transform, in Hankel transform is played by ajinc-
/unction ([17]) that, similarly to sinc-function as a Fourier transform of a
rect-function (Eq. 2.4.8), is Hankel transform ofthe rect-function:

""
jinc{r)= 2n frect{/Yo{2n/r)d/ . (2.4.38)
o

Sinc- andjinc -functions are compared in Fig. 2-16. As one can see from the
figure, jinc function oscillates much less and converges to zero much faster
then sinc function.
48 Chapter 2

sinc(x)

O.
,
O. .........................;............................................................. f·!~ ......·........·....·.................·............ · ........ ·<......•........ ·........ ·........ij
O. .. .... ·........+.. ·.... ·........ ·.... ·........ ·+......·..·.. ·............·.. t ..;...·......·.................... + ............................... ................................-iI
~

O.

0 ..........,.,.,..,."..,,--

-0.2.""===="",===="""===,,,,'9t'''''''"'===-========,",,,
-60 x 60
jinc(x)

0.8~1 ............................ !.................... + ............. ·.. ··· .... ·····IH··· .. ····· .... ······ .. ···!···················· ...... ·" .. ·· .. ·· .. ••·· .. · .... ··4
0.6~i····· .... ·.. ·.. ······..···· ..,·· .. ···················+· .... ····· .. ·· ....······l·H· .. ·.... ·..........·........ ·!.... ·· .... ·· .. ······ .. ··t .. ····· .. ·.. ·· .. ·.. ·· .. ···4
0.4 1;- ................................ ;.................................,........................... ·1 .. ·;·+· ..........·.... ··........ ·; ...... ·...... ·........ ·...... ·.. ".............. ·..·.............;1
0.21;-1 ................................ ;.................................;...........................1....;.. +...........................;................................"................................;1

o -------~""'~:(; ':(\T''''"~---------iI
j;-: .. ...

-0.1_ 60
-40 -20 o 20 40 x 60

Figure 2-16. Sine- and jine- functions

2.4.2.4 Mellin trans form

The shift invariance of the Fourier transform may be tumed into ascale
invariance property if one introduces a logarithm scaling of the signal
coordinate:

x=lnz (2.4.39)

Substituting Eq 2.4.39 in Eq. 2.3.l7a yields:

a (f) = }a(ln z)exp(i21tf In z) dz = }a(ln z)zi21t/-t dz . (2.4.40)

o z 0

This brings us to a trans form:

fa (Z)Zi2lt
00

J.l(f)= f -t dz. (2.4.41)

 2. Optical signals and trans/arms 49

referred to as the Mellin trans/orm. For Mellin transform, scaling signal

coordinate z = kz modi fies its Mellin spectrum in the same way as signal
shift modi fies Fourier spectrum (see Eq.2.4.1O):

/l(/) = kfa (kzXkzt n/- 1tfi = ki2n/ fa (kzXzt

00 00

n/- 1 tfi = ki2n/ fl(/).

o 0
(2.4.42)

Therefore, Mellin spectrum of a signal, by its absolute value, is invariant to

scaling signal coordinate.
Two dimensional Mellin Transform is generated by the 2-D Fourier
transform where coordinate conversion (2.4.40) is applied to both
coordinates and, by this definition, is separable over two variables:

(2.4.43)

More details regarding Mellin transform one can find in [18].

2.4.3 Integral Fresnel and related transform

2-D integral Fresnel transform was introduced in Sect. 2.3.2 (Eq. 2.3.17)
as a mathematical model that approximates Kirchhoff integral in near zone
diffraction. l-D Fresnel transform is defined respectively as

(2.4.44)
-00

Fresnel transform and Fourier transform are very c10sely connected and
may be reduced one to another by mean of phase modulation of signal and
its spectrum with chirp-functions functions chirp(x)= exp(- irrx 2 ) and
chirp(f)= exp(- i1tf2) :

(2.4.45)
-00

From the invertibility of the Fourier transform it follows that inverse

Fresnel trans form can be obtained as:
50 Chapter 2

a(x)= exp(i1tX 2)}a Fr (j)exp(i1tf2 )exp(- i2fx}Jf =

~!! Ja (I )exp[i1t (X - 1 t ~I = }a (I )exp[i1t (X - 1 t ~I .

Fr Fr
-F
(2.4.46)

Therefore, the kerne 1 of the Fresnel transform is self conjugate and his
corresponding delta function is:

8 (x,~)= ~2!exp[i1t (x 2 -~ 2)] fexp[-i21tf(x-~ )]df =

-F (2.4.47)
exp[i1t (x -~ 2)].lim{2F
2 sinc[21tF(x -~ )ll.
F->oo

The result ofFresnel transform of a rect-function

frinc(x; F) = }reet ( f + F ) exp[i1t (x - f Y~f =

-00 2F
F
exp(i1tX 2 ) fexp~1t/ 2 )exp(- i21tfx }Jf (2.4.48)
-F

is an analog of sinc-function of Fourier transform (Eq. 2.4.7). We will refer

to it as to frine-funetion. Frinc-function is aversion of a function

1 !i.
Fr± (z)= FrC(z)± FrS(z)= ~ fexp{± ix 2 ~ (2.4.49)
,,21t 0

known in mathematicalliterature as Fresnel integral ([19]). The relationship

between frinc-function and Fresnel integral is described by the formula:

Fresnel transform can also be regarded as a convolution transform and it

can also be linked with Fourier transform through the convolution theorem.
Fresnel transform a Fr lf) of a signal alx) can be found as inverse Fourier
transform
 2. Optical signals and trans/arms 51

fa
00

a Fr (J) = F (p )CHIRP(p )exp(- i2rr.JjJ )dp (2.4.51)

of a product of signal Fourier spectrum a P (p) and Fourier transform

CHIRP(p )of a chirp-function which is, to the accuracy of irrelevant
constant, also a chirp-function ([19]):

CHIRP(p)= }exP(-irr.x2)exP(i2rr.PX~= l~i exp(irr.p 2)= l~i chirp(ip).

-00 -V 2 -V 2
(2.4.52)

Two dimensional analogs of Eqs. 2.4.45 and 46 follow straightforwardly.

Integral Kirchhoff transform (Eq. 2.3.16) whose approximation is Fresnel
transform is also a convolution and can also be represented via Fourier
transform:

f fa
00 00

p (px,py)K(px,py)exp[-i2rr.{txpx + fypy)~Pxdpy ,(2.4.53)

-00 -00

where

~ ~ eXP{-i.Jdf +d,[(x- fxl' +(y- fylf}

K{px,py)= ff x
-00 -00 ~d~ + d,j{x - fJ2 + (y - fy)f
expl- i2rr. (JxPx + fypy )}txdy (2.4.54)

The result of Fresnel transform of an arbitrary function a(x) is, generally,

a complex valued function. Its absolute value can be interpreted as the
magnitude of the wave front and its phase as the phase of the wave front in
near zone of diffraction of wave front a(x). Physical sensors are insensitive
to the phase of wave fronts and measure only its magnitude. Therefore one
may want to consider a reduced, or partial Fresnel transform

1
CiPr (J) = a(x )exp(- irr.x 2 )exP(i2rr.fx ~
-00
(2.4.55)
52 Chapter2

Partial Fresnel transform is Fourier transform of a signal modulated by a

chirp-function. Its inverse transform is simply the inverse Fourier transform
phase modulated with a chirp-function:

(2.4.56)
-00

Fresnel transform as a mathematical model of transformation in optical

systems can be regarded as a special case of the so called ABCD-transform
([20-22]):

(lABCD(f)=cIa(x)exp[~ (Ax l - 2.fx+ Df 1 )}a. (2.4.57)

When A = D =0 and B =1, ABCD-transform is reduced to Fourier

transform, when A = B = D =1, ABCD-transform is reduced to Fresnel
transform. The special case of A = D = cos cp and B =sin cp is called
Fractional Fourier transform:

(lF,F(f) = cexp (i1t ~)ja(x)exP(i1t ~)exp[i:1C.fx]dx .(2.4.58)

tancp _oe> tancp smcp

Originally introduced in quantum physics ([23]), it was later found to be

convenient for describing optical phenomena such as wave propagation in
graded index optical fibers ([24-27]).
From the fact that ABCD-transform is reducible to Fourier transform:

(2.4.59)

it follows that its is invertible and his kernel is self conjugate:

1
a(x)=- f(lABCD(/)exp -i1t
<Xl [Ax 1 - 2.fx + Dfl ] d/. (2.4.60)
Bcl B
 2. Optical signals and trans/orms 53

2.4.4 Hilbert transform

Hilbert transform represents yet another signal transform of the

convolution type closely linked with Fourier transform. Hilbert transform
deals with the redundancy associated with the symmetry of Fourier spectra
ofreal valued signals (Eq. 2.4.18).
Let a(f)=a ,(- I) is Fourier spectrum of a signal a(x). Form an
auxiliary signal a(x) by cutting out frequency components of signal a(x)
for negative frequencies and, in order to preserve signal energy, multiplying
its spectrum on positive frequencies by a factor of2:

a(x) = 2 Ja (I )exp(- i2nfx }JI = Ja (I Xsign(/) + 1]exp(- i2rifx}JI ,

00 00

o
(2.4.61)

where sign(/) = III = {+ 1, I ~0

I -1, I< 0
Signal a(x) can be represented in a form of two components:
a(x) = a(x) + i(x) , where i(x) is

i(x)= Ja(/)sign(/)exP(-i2nfx}JI. (2.4.62)

Because spectrum a (I )sign(/) IS anti-symmetrical:

a (I )sign(J) = a . (- I )sign(- I), for real valued signal a(x), real part of
a(x) is equal to a(x) and i(x) is pure imaginary. Real valued signal a(x)

a(x)= ~i(x)= ~ fa (/)sign(/)exp(- i2nfx}JI . (2.4.63)

I I -00

is called Hilbert transform of a(x) ([3]).

Signal

a(x) = a(x)+ ia(x) (2.4.64)

is called analytical signal for real valued signal a( x). Fourier spectrum of
analytical signals is nonzero only for positive frequencies.
54 Chapter 2

From Eq. 2.4.64 and by virtue of the convolution theorem it follows that
Hilbert transform is a signal convolution with PSF defined as

h(x)=~ fSign(/)exP(-i21t.fx)d/. (2.4.65)

1 _00

To find this integral, one can use identity ([28]):

OOfsinmx dx =-slgn
1t . (m ) (2.4.66)
o x 2

and modify it in the following way:

1 -00 x
j
2 -00 x
1 2 -00 x
1
sin 2rrfx dx = sin 21t.fx dx =! sin 21t.fx dx +! COS 21t/x dx =
o x

~ oof exp(i21t.fx) dx = ~ sign(/ ~ . (2.4.67)

21 -00
x 2

Then taking inverse Fourier transform, obtain:

~
'_
fsign(/)exp(- i21t.fx)d/ = - ~.
u
(2.4.68)

Therefore, Hilbert transform, as a convolution integral, is defined as

a(x)=! f a(~) d~ . (2.4.69)

1t -oo~ - X

Because ~-l sign / Xr

1 sign /)= -1, inverse Hilbert transform is
identical to the direct one except for the sign:

Note that these definitions are direcdy linked with the definitions of the
direct and inverse Fourier transform (Eqs. 2.4.1 and 2.4.3). If in these
definitions sign of exponents are opposite (see, for instance, [3]) , direct and
inverse Hilbert transforms also will also change their signs.
 2. Optical signals and trans/orms 55

One of the most immediate applications of Hilbert transform is

determination of signal's complex amplitude, signal's envelope and phase.
Signal's envelope is absolute value ofthe corresponding analytical signal:

(2.4.71)

Such name is justified by the fact that A(x);;:: /a(x ~ and that functions A(x)
and a(x) never intersect and have common tangents wherever they touch
each other:

~A(X~ =~a(x) (2.4.72)

dx t(x)=a(x) dx

The phase of the analytical signal is defined as

8 (x)= Im(ln a(x ))= arctan[ ~~= lJ (2.4.73)

and complex amplitude is defined as A(x)= A(x )exp(i8 (x)).

The physical meaning of the envelope and phase of analytical signal can
be illustrated on an example of sinusoidal functions

(2.4.74)

with a carrier frequency 10 variable amplitude ao(x) and phase 8 . Such

functions are, for instance, mathematical models of signals in holography
and interferometry. Rewrite Eq. 2.4.74 in the form:

a(x) = ao(x )cos8 (x )cos 2rt/ox - ao(x )sin8 (x )sin 2rt/ox . (2.4.75)

Because Fourier spectra of functions cos 2rt/ox and sin 2rtfox are,
respectively, [8 (I - 10) + 8 (I + 10 )]/ 2 and [8 (I + 10) - 8 (I - 10 )]/ 2i ,
their Hilbert transforms are, respectively, - sin 2rt/ox and cos 2rt/ox. Then,
in the assumption that Fourier spectra of functions ao(x )cos8 (x) and
ao (x )sin8 (x) do not extend outside interval [- 10,/0]' Hilbert transform
of a(x) is:
56 Chapter 2

a(x)= -a o(x )cos9 (x )sin 21t/ox - ao(x )sin9 (x )cos 2rtfox =

(2.4.76)
- ao(x )sin[2rtfox + 9 (x)]
and its analytical signal is

a(x)= ao(x )cos[21t/ox -9 (x )]- iao(x )sin[2rtfo x -9 (x )]=

ao(x )exp[- (i21t/ox -9 (x ))]= (2.4.77)
ao (x )exp(i9 (x ))exp[- i21t/ox] = A(x )exp[- i21t/ox]

where A(x)= a(x )exp(i9 (x)), the form adopted for representation of
sinusoidal signals on a complex plain.
 2. Optical signals and trans/arms 57

2.5 IMAGING FROM PROJECTIONS: RADON AND

ABEL TRANSFORMS

One of the most important recent findings in image science resulted in

various methods of tomography is the discovery that image can be
reconstructed from its projections. Signal formation by the procedure of
image projecting with parallel beams is described by Radon transform.

Parallel X-ny
beam y

sensitive sensors

s
Figure 2-17. Parallel beam projection tomography

Radon transform defines projection of 2-D signal a{x, y) given in a

rectangle coordinate system {x, y) onto an axis ~ rotated with respect to
axis x by the angle S (Fig. 2-17) as:
 58 Chapter 2

cr (S,S)= f fa(x,y)s (S - xcosS - ysinS )dxdy (2.5.1)

Xy

Projection signal cr ts ,S) is called sinogram.

Radon transform and Fourier transforms of projections are linked with
projection theorem:
1-D Fourier transforms a s (I) of object projections cr (S,S) are cross-
sections of 2-D object spectrum taken at the corresponding angles.
Proof
00

alt (j)= fcr (S,S )exp(i21tA)/s =

-00

1{f fa(x,
-00 XY
y)s (S - x cosS - y sin S )dxdY} exp(i21tA)/s =
(2.5.2)

f fa(x,y )exp[i21t C/XcosS + lYsinS )}txdy =

XY
a (I cos s, 1 sin S )
where a (Ix' I y ) is 2-D Fourier spectrum of a(x, y).
From the projection theorem it follows that inverse Radon transform can
be computed as 2-D inverse Fourier transform ofprojections' spectra a s (I)
in polar coordinate system (/,S):
n 00

a(x,y)= f fas(/)exp[-i21t(jcosS + IsinS)]dSdl=

-1[-00
(2.5.3)
f fa (I cosS,1 sinS )exp[- i21t (I cosS + 1 sinS )]dSdl
n 00

-1(-00

This method of computing inverse Radon transform is called Fourier

reconstruction method. Yet another method of inverting Radon trans form,
Jiltered back projection method, is also derived from the interrelation
between Fourier and Radon transforms.
2-D function a(x,y) can be found from its 2-D Fourier spectrum
aVx,ly) as

(2.5.4)
 2. Optical signals and transforms 59

In polar co-ordinate system (w ,9 ) in frequency domain (Fig. 2-18),

/x = wcos9; I y = wsin9; d/xdly = Iwldwd9 ; 9 E [O,1t], (2.5.5)

obtain:

" ""
fd9 flw/a{w,9 )exp[- i21tw(xcos9 + ysin9 )}lw =
o
(2.5.6)
"
fFBproj(xcos9 + ysin9)1e,
o

where

FBproj(xcos9 + ysin9)=
oe
(2.5.7)
flw/a(w,9 )exp[- i21tw(xcos9 + ysin9 )}lw .
-00

a(w,e)

Figure 2-18. Cartesian and polar coordinate systems in Fourier transfonn domain

Eq. (2.5.6) describes reconstruction of object signal atx, y) as a "back

projection" of signal FBprojtxcos9 + ysin9) of Eq. (2.5.7) that is
obtained as inverse Fourier Transform of 1-0 spectrum atw,9 ) modified by
a filter with frequency response IWI ("ramp filter"). Spectrum atw,9) is a
cros-section of the object's 2-0 spectrum along straight line that goes under
angle 9 . According to the projection theorem, it is 1-0 Fourier Transform
the corresponding projection of the object. Therefore, Eq. (2.5.6) represents
60 Chapter2

reconstruction of object signal atx, y J as a result of accumulating "back

projections" of its "filtered projections" FBprojtx cose + y sine J.
Obviously, exact inversion of Radon transform assumes infmite number
of projections. In practice, only finite number of projections is always
obtained. Although reconstruction methods can, in principle, be
implemented in optical setups, the reconstruction in most applications is
carried out in a digital form in computers. Some computer algorithms of
tomographie reconstruction are described in Chs. 8 and 9.
For circularly symmetrie signals a(x, y)= a(~ x 2 + y2 ), Radon transform
is reduced to Abel transform:

(2.5.8)

or, in polar coordinates (r,S)

(2.5.9)

Abel transform is naturally linked with Hankel and Fourier transforms. In

particular, Abel transform ofthe jinc-function is the sinc-function

2} Ji~c(r )2 rdr = sinc(x). (2.5.10)

x r-x

More details regarding properties of Radon and Abel transforms can be

found in Refs. [17,29,30].
2. Optical signals and transforms 61

2.6 MULTI RESOLUTION IMAGING: WAVELET

TRANSFORMS

Above described convolution and Fourier and related orthogonal

transfonns describe wave propagation and image fonnation in a large variety
of imaging devices. It is frequently convenient to regard applying these
signal transfonns to signals a{x) taken at an arbitrary scale s in coordinate
sx. Because kernel of Fourier and related transfonns is a function of a
product of coordinates in signal and transfonn domain, these transfonns can
readily be treated as transforms of signals in scaled coordinates with the
scale factor being a coordinate in transform domain. This is not the case for
convolution transforms. Applying the idea of signal treatment in scaled
coordinates to convolution leads to a new type of integral transfonns that
was given name of wavelet trans/ornis ([31-34]).
Consider convolution integral in a form
00

a(x)= fa(~)h(x-~~. (2.6.1)

Let now signal a(x) is considered in a scaled coordinate system sx. Then
its convolution with PSF h() depends on both sand x :

00

a(s;x)= Ja(s~)h(x-~~. (2.6.2)

In Eq. 2.6.2 signal is taken at different scales and convolution kernel is kept
the same. One can now translate signal scaling into convolution kernel
scaling by changing variables s~ ~ ~ and obtain:

a(s;x)=! Ja(~)h(sx -~)~ (2.6.3)

s -00 s

or, in scaled coordinates sx ~ x :

s
J
a(s; x) =! a(~ )h( x - ~ )~ .
-00 s
(2.6.4)

Eq. 2.6.4 is commonly accepted as adefinition ofwavelet transfonn. In such

a representation it can be regarded as signal analysis with PSF taken at
different scales, or signal mutiresolution analysis.
62 Chapter 2

From the point of view of classification of integral transforms, wavelet

transform can be given yet another interpretation. Different transforms can
be classified in terms of trans form kernel. All trans form kerneis are built
from a single mother function by its modification parameterized in terms of
transform domain coordinates. Convolution integrals represent transforms
with kerneis obtained from a mother function by its translation or shift. In
Fourier and related transforms kerneIs are built by means of scaling
coordinates of the mother function. Wavelet transform are built on the
principle of combination of shift and scaling. Such classification helps to
understand approaches to the design of transforms for signal discretization
and digital processing (see Sect. 3.2).
In I-D wavelet transforms, transform domain argument of the kernel
function has two components: shift ~ and scale s. There might be many
wavelet transforms that differ one from another in selection of the mother
wavelet h(.). This selection is the main issue in the design of wavelet
transforms and is governed by the requirement of the invertibility of the
transform and by implementation issues.
As one of the immediate examples of the wavelet transforms one can
consider Fresnel Transform in non-reduced coordinates

(2.6.5)

with scale parameter .J').J) (see also Eq. 2.3.10). Haar Transform suggested
in 1910 ([35]) is yet another example of a wavelet trans form. Kernel of Haar
transform is abipolar step function

h(s;x)= reet( 2; J- reet( 2Xs-l J. (2.6.6)

By the present time, wavelet transforms have become a very popular tool
for signal processing and quite a few of different wavelet transforms have
been suggested ([33]). Two dimensional and multi-dimensional wavelets are
usually defined as separable functions.
2. Optical signals and transforms 63

2.7 SLIDING WINDOW TRANSFORMS AND "TIME-

FREQUENCY" (SPACE-TRANSFORM) SIGNAL
REPRESENTATION

2.7.1 Transforms in sliding window (windowed transforms)

Fourier integral transform and its derivatives are all defined with respect
to signals defined on infinite intervals. In reality, however, only a finite
fraction of the signal selected by the signal sensor aperture is or should be
analyzed. This can mathematically be modeled by signal windowing. In this
way we arrive at windowed transforms:

J
00

a(x, f)= w(~ )a(x -~ )P(f,~ ~ (2.7.1)

where <p tf,~) is a transform kerne I and wt~) is a window function such
that it is elose to I within a certain vicinity to x =0 and then more or less
rapidly decays to zero:

w(O) = 1; limw(x)=O
x->oo
(2.7.2)

Windowed transforms of signal atx) are defined for every point x of signal
coordinates and usually are computed in a process of regular signal scanning
along its coordinate. In this sense they will be referred to as sliding window
transforms.

2.7.2 Sliding window Fourier Transform

Sliding window Fourier transform (SWFT) is one of the most important

sliding window transforms. It is defined as

00

a(x,f)= Jw(~)a(x-~)exp(i21tft.,~ . (2.7.3)

SWFT of 1-0 signals is a function of two variables: coordinate in signal

domain and frequency in Fourier domain. It represents what one can call
signal loeal speetrum; that is, spectrum of a (windowed) signal segment
centered at the center of the window. This spectrum regarded as a function
of coordinate x that defines running position of the window and frequency
64 Chapter 2

f is ealled signal space-frequency representation (time-frequency

representation, in eommunieation theory jargon).
From inverse Fourier transform of the windowed Fourier transform
a.
speetrum (x, j) obtain:

Ja. (x, f )exp{- i21t.ft}tJj .

a;,

w{1; )a{x -I; ) = (2.7.4)

-a;,

By virtue ofEq. 2.7.3 it follows from Eq. 2.7.4 that

00

a{x)= Ja.{x,j}tJj. (2.7.5)

Eq. 2.7.5 ean be regarded as inverse SWFT.

Window funetion of SWFT ean be an arbitrary funetion that satisfies Eq.
2.7.4. Seleetion of the window funetion is governed by the requirements to
Ioeal speetrum analysis. SWFT with window funetion

(2.7.6)

with a as a window width parameter is known as Gabor transform ([7]).

2.7.3 Sliding window and wavelet transforms as signal sub-band

decomposition

One ean give to Windowed Fourier Transform one more and very
instruetive interpretation. One ean regard Eq. 2.7.3 as a eonvolution integral
of a{x) with kernel w{1; )exp{i21t.ft ). Therefore, aeeording to the
eonvolution theorem, Fourier transform of the signal time-frequeney
representation a.{x,f) at frequeney j

Ja. (x, f )exp{i21tpx}dx

00

A{p, f) = (2.7.7)

is a produet A{p,f)=a.{p )f'Ap) ofsignal Fourier speetrum

 2. Optical signals and transforms 65
<Xl

a(p)= fa(x)exp(ilnpx)u- (2.7.8)

and Fourier Transform W f (p) of the convolution kerne I w(~ )exp(ilnA) .

The latter is nothing but shifted by 1 Fourier spectrum of the window
function w(~):

<Xl

Wf (p) = f[w(~ )exp(ilnA )]exp(ilnp~ ~ =

<Xl
(2.7.9)
fw(~)exp[i2n(p+ I)~ =W(p+ I)

Therefore, signal time-frequency representation a(x, I) at frequency 1 can

be regarded as signal's sub band formed by signal filtering by a filter with
frequency response W(p + I).
Similar is true for a general sliding window transformation (Eq. 2.7.1):
sliding window signal transformation is a signal's sub-band formed by a
filter with frequency response equal to Fourier Transform of the
corresponding windowed basis functions:

(p) = fw(s )p (/,s )exp(ilnps)ts .

<Xl

Wf (2.7.10)

Because wavelet transforms are, for every scale, signal convolution they
can also be treated as signal sub-band decompositions formed by signal
filtering by filters with frequency response equal to Fourier Transform ofthe
mother function on the corresponding scale:

Ws (p) = }p (: Jexp(i2nps ~ . (2.7.11)

This property establishes a link between wavelet and sliding window

transforms. Figs. 2.19 and 20 illustrate arrangements of sub-bands for sliding
window Fourier transform and wavelet base known as Binom5, respectively.
As one can see from the figures the main distinction between these two sub-
ban decompositions is that sliding window Fourier transform decomposes
signal into sub-bands of the same width uniformly arranged in the base band
while sub-bands for wavelet decomposition are arranged in logarithm sc ale
and have width that is doubled with the increase of its frequency.
66 Chapter 2

05
O~~O-I--~~----~"~

Figure 2-19. Arrangement ofsignal sub-bands for sliding window Fourier Transform

: :

Figure 2-20. Arrangement of signal sub-bands in wavelet BinomS decomposition

2. Optical signals and transforms 67

2.8 STOCHASTIC TRANSFORMATIONS AND

STATISTICAL MODELS

Description of mathematical models of signal transformations will not be

complete without considering statistical approach to signals and
transformations. As it was mentioned in Sect. 2.1.2, results of signal
processing and performance of imaging and measurement systems is usually
evaluated on average. For statistical averaging, signal transformation
characteristics such as, for instance, point spread function and frequency
response, are, similarly to the signals, also modeled as random values or
functions. In addition, one would need to statistically describe interaction
between signals and transformation systems. The most useful models to
describe the interaction are those of additive, multiplicative, Poisson noise,
impulse and speckle noise models.

2.8.1 Additive, multiplicative, Poisson noise and impulse noise

models

The model of additive signal independent noise model (ASIN-model)

assumes signal transformation described as

b(x)=a(x)+n(x), (2.8.1)

where a(x) is input signal, b(x) is output signal and n(x )is a random
process statistically independent on signal a(x) and referred to as additive
noise. The assumption of statistical independence assumes that statistical
averaging applied to signal b(x) and to results of its transformations can be
carried out on statistical ensembles of signals {a{x)} and noise {n{x)}
separately:

A V{a(x )},{n(x)} {b(x )} = AV{a(x)} {a(x )} + AV{n(x)} {n(x )} . (2.8.2)

For ASIN-model, mean value n = AV{n(x)dn{x)} of noise is of no

concem and it is natural to assume that ii = O. With this assumption, mean
value of the signal corrupted with additive zero mean noise is equal to the
uncorrupted signal and standard deviation of its fluctuations is equal to
standard deviation of noise:

AV{n(x)} {b{x)} = a(x ~. (2.8.3)

68 Chapter2

(2.8.4)

For ASIN-model, ratio of signal mean value to standard that characterizes

intensity of signal fluctuations with respect to signal mean value is
proportional to signal mean value.
Typical statistical model of additive noise is Gaussian random process
([36]). Gaussian random process is completely statistically determined by its
probability density called normal distribution

p(n)= ~ exp[- (n - ~)2], (2.8.5)

21t0' n 20' n

where 0' n is its standard deviation and ii is its mean value, which, for
ASIN-model, is usually assumed to be zero, and by its correlation function

(2.8.6)

According to the centrallimit theorem of the probability theory, normal

distribution is a good model for probability density distribution of random
values that are obtained as a sum of very many statistically independent
random variables. For instance, it is a good model of the distribution of
thermal noise caused by random fluctuations of velocities of electrons in
electron current in signal sensors.
Random process is a stationary process, if its correlation function
depends only on distance between points Xl and X2:
Rn (Xl' X2 ) = Rn (Xl - X2 ). Fourier transform of correlation function of a
stationary random process is called its spectral density, or power spectrum:

f
00

N(f) = Rn (X )exp(i21tjX )Ix . (2.8.7)

-00

Altematively, spectral density of a stationary random process can be defined

as average, over all realizations of the process {n(x)}, of squared module of
Fourier spectra ofthe realizations

(2.8.8)
2. Optical signals and trans/orms 69

Random process is called uncorrelated if its spectral density is uniform

function, or respectively, its correlation function is delta-function:

N(/)= No =a;;
(2.8.9)
Rn (x) = Noo (x)=a;o (x~

Uncorrelated Gaussian random process is conventionally called white noise.

In a number of applications, additive noise is highly correlated. The
correlation is most convenient to describe as concentration of noise power
spectrum in certain areas of frequency domain. If noise power spectrum is
concentrated only around a few isolated points in frequency domain, this
type of additive noise is called moire noise.

Multiplicative noise (MN-) model assumes signal transformation defined

as

b(x) =m(x). a(x) , (2.8.10)

where m(x) is signal independent random process with mean equal to 1.

Similarly to mean value of signal corrupted by additive zero mean noise,
mean value of a signal corrupted by multiplicative noise is also equal to the
mean value ofnon-corrupted signal:

AV{b(x )}= AV{II(x)} {a(x)}. AV{n(x)dn(x )}= AV{II(x)} {a(x)}. (2.8.11)

Variance of the corrupted signal is proportional to the variance of non-

corrupted signal and variance of noise:

(2.8.12)

For multiplicative noise, ratio of corrupted signal mean value to its

standard deviation is equal to standard deviation of the multiplicative noise
and does not depend on the signal mean value. This is a characteristic
property of multiplicative noise.

Poisson noise model

Poisson noise model describes quantum fluctuations in sensors such as
fluctuations of number of electrons in CCD or CMOS photo and X-ray
sensitive sensors. For Poisson noise model, signal values are nonnegative
integer numbers that have Poisson distribution probability density:
70 Chapter 2

-q
p(q)= L exp(- q), (2.8.13)
q!

where q is mean value of q. It is the property of Poisson distribution that

standard deviation of q is also equal to q:

(2.8.14)

For q ~ 00, Poisson distribution, according to Moivre-Laplace theorem,

tends to normal distribution with standard deviation .Jif

(2.8.15)

For Poisson distributions, ratio of standard deviation to mean value tends to

zero with growth of the mean value:

q-'>oo ) 0, (2.8.16)

which means that with growth of q fluctuations are becoming less and less
noticeable.

Impulse noise model (ImpN-) model

b(x)= [1- e(x )]. a(x)+ e(x )n(x) (2.8.17)

assurnes that noise is caused by two random factors: random binary process
e(x) that takes two values, zero and one, and an additive noise n(x). ImpN-
model is applicable to discrete signals. It is characteristic for signal
transmission systems with random faults. The process e(x) is specified by
the probability that e(x) = 1 (the probability of error, or the probability of
signal loss). Statistical properties of noise n(x) can be arbitrary. If random
process n(x) takes only two values equal to minimum and maximum of
a(x), it is, in image processingjargon, referred to as pepper and salt no;se.
 2. Optical signals and trans/orms 71

Justification for using ASIN-, MN- and ImpN-models and statistical

characteristics of noise ensemble should be based on the design
characteristics of the sensors, imaging and image transmission systems.
Examples of images distorted with additive and impulse noise are shown
in Fig. 2-21. The noise level in images is characterized by the ratio PSNR of
image dynamic range to standard deviation of noise. For impulse noise,
standard deviation of noise is computed as standard deviation of difference
between initial and noisy images.

Additive Gaussian noise,

oise less image

Additive "moire" noise, Impulse noise, probability of error

PSNR==20/256 0.06, PS R==20/256

Figure 2-21. Examples ofimages with additive and impulse noise. Observe different visibility
of different types of noise with the same standard deviation
72 Chapter2

2.8.2 Speckle noise model

Phenomenon of speckle noise is characteristic for coherent imaging

systems such as holographie ones, synthetic aperture radar and ultrasound
imaging systems. Speckle noise originates from property of objects to
diffusely scatter irradiation and is caused by distortions introduced by wave
front sensors. Conventionally it is associated with finite resolving power of
sensors, or, in holographie systems, by the limitation of the area in which
wave front is measured by sensors or recorded on holograms ([37]).
Let a(~,,, )exp[ie (~ ,,, )] be complex amplitude of wave front in the
object plane that describes object reflectance/transmittance properties. For
describing property of objects to diffusely scatter irradiation, e (~,,,) can be
modeled as a random process. Diffuse uniform in aB direction scattering
correspond to spatially uncorrelated process e (~ ,,, ). Specular scattering
takes place if e (~ ,,,) is highly spatially correlated process.
Consider coherent imaging system with point spread function
h(x,y;~,,,). Then, for input object a(~,,,) exp(ie(~,,,)), output image
formed by a sensor that is sensitive to squared magnitude of the output wave
front is

(2.8.18)

Let the object has a uniformly painted surface such that a(~,,,) = A o.
Then:

b' (x,y l+.11exp[;e (I;,~ l]h(x,y; ,~ lt/S I' ~ Ibnl' +"'1'

I; dt]

(2.8.19)

where

b = A o f fcos~ (~,,,)]h(x,y;~,,,)~ d" ,

'" '"
fe (2.8.20)
-00-00

bim =Ao f fsin~(~,,,)] h(x,y;~,,,)d~ 4rl

'" ""
(2.8.21)
-00-00
2. Optical signals and transforms 73

If functions cos[e (I;, TI)] and sin~ (I;, TI)] are changing much faster then
point spread function h(x,y;I;,T1), or, in other words, many uncorrelated
values of exp[ie (I; ,TI)] are observed within aperture of the imaging system,
the central limit theorem of the probability theory can be used to assert that
b,e and b im are normally distributed with zero mean:

b'" = AVa(b"')= Ao f fAVa {cos[e (~,11 )]}h(x,y;~,11)~ d11 =0 ;

b1m = AVa(b im )= Ao f fA Va {sin[e (~,11 )]}h(x,y;~ ,11)~ dT) = 0 •
(2.8.22)

Find correlation function of orthogonal components b,e and b im • For b,e,

A; JJAVa {cos[e(~I)Jcos~(~z)]}h(Xd~l)h·(Xz;~z)~1 ~z =
<Xl <Xl

-co-co

A; J JR9(~I'~z}h(XI;~I)h·(Xz;~z)~1 ~2'
<Xl <Xl

(2.8.23)

where x=(x,y), ~=(I;,T1) and Ra(~I'~z) is correlation function of

cos~ (~,.., )]. In the above accepted assumption of applicability of the central
limit theorem, Ra (~I ,~J can be regarded, with respect to integration with
h(x,y;~ ,TI), delta function which implies that

Rb" (Xl ,X 2 )= -i Ai jh(X ;~)h (x ;~~.

-00
I * 2 (2.8.24)

In the same way one can obtain that:

R bl.. (XI ,X z )= Rb" (XI ,X z)= -i A; fh(X I;~)h· (xz;~~·

-<Xl
(2.8.25)

From Eqs. 2.8.25 and 26 it follows that variances of orthogonal components

b,e and b im are
74 Chapter 2

a: =(breY
-- -- 1
=(b im )2 ="2 A ; JJlh(x,Y;~,l1Wd~d11'
0000

-00-00
(2.8.26)

In general, they are functions of coordinate (x, y) in image plane. For space
invariant imaging system (h(x, y;~, 11) = h(x - ~,y -11)) they are coordinate
independent:

a; = i A~ Ill h(x,y)2 dxdy (2.8.27)

-00 -00

Having defined probability density function of orthogonal components of

b 2(x,y) one can now find prob ability density of b 2(x,y) as the probability
density of sum of squared independent variables b,e and b im with normal
distribution. To this end, introduce random variables Rand S such that

(2.8.28)

Probability that b,e and b im take values within a rectangle d( b,e )d( b im )
is

d9 R
--exp
21t a b2
(R
---
2cr b2
2
JdR=---exp
d9 1
21t 2cr
--- JdR
(R
2cr 2
2

2
2 . (2.8.29)
b b

This is joint probability of variables {R 2 ,S } . It follows, therefore, that

{R = b(x,y)}
2 and S are statistically independent, S is uniformly
distributed in the range {O,21t} and b(x, y) has exponential distribution
density:

P(b) = _1_ exp(- _b_J ' (2.8.30)

2a; 2cr;
 2. Optical signals and transforms 75

where 0': is defined by Eq. (2.8.27).

From Eqs.2.8.19 and 27 one can immediately see that mean value of
b 2 (x,y} is:

=A; f flh(x, yf dxdy

<Xl <Xl

b 2 (x,y}=2cr; (2.8.31)
-00-00

It is the property ofthe exponential distribution that its standard deviation

is equal to its mean value

0' 62 = (r b 2(x,y}r Y/2 = A; f flh(x,yf dxdy .

-00-00
(2.8.32)

Fluctuation of b 2 (x,y) around its mean value are called speckle noise. It is
customary to characterize intensity of speckle noise by ration of its standard
deviation to mean value called "speekle contras/':

0'2
Spekl eontrast = 6 • (2.8.33)
- b 2 (x,y)

It follows from Eqs. 2.8.31 and 32 that, for objects that scatter irradiation
almost uniformly in the space, speckle contrast of their image obtained in
coherent imaging systems is unity:

Spekl _ eontrast = 1 . (2.8.34)

Therefore speckle noise can be regarded to be multiplicative with respect to

image mean value. Fig. 2-22 illustrates appearance of speckle noise and its
spatial correlations that depend on imaging system point spread function
(Eqs. 2.8.24 and 25).
In general, speckle noise in coherent imaging systems appears not only
owing to insufficient resolving power of the systems. In fact, any distortions
of the wave field that may happen in the sensor such as limitation of the
signal dynamic range or signal quantization for its conversion into a digital
signal for signal processing also cause appearance of speckle noise. These
phenomena are discussed in Sect. 7.3.
76 Chapter 2

a) h)

Figure 2-22. Images with speckle noise for imaging systems with high (a) and low (h)
resolving power.
 2. Optical signals and transforms 77

References

1. W. B. Davenport, W. L. Root, An introduction to the theoty ofrandom signals and noise,

McGraw-HiIl, N.Y., 1958
2. B. R. Frieden, Numerical Methods in Probability Theoty and Statistics, in: Computers in
Optical Research, Methods in Applications, B. R. Frieden, ed., Springer Verlag, N.Y.,
1980
3. A Papoulis, Signal Analysis, McGraw-Hill, N.Y., 1977
4. J. M. Wozencraft, I. M. Jacobs, Principles of Communication Engineering, J. Wiley &
Sons, N. Y., 1965
5. L.P. Yaroslavsky, The Theoty of Optimal Methods for Localization of Objects in
Pictures, In: Progress in Optics, Ed. E. Wolf, v.XXXII, Elsevier Science Publishers,
Amsterdam, 1993
6. D. Gabor, A new microscopic principle, Nature, v. 161,777-778,1948
7. D. Gabor, Theoty ofCommunication, Journ. OfIEE, vol. 93,1946, pp. 429-457
8. E.N. Leith, 1. Upatnieks, New techniques in wavefront reconstruction, JOSA, v. 51,
1469-1473,1961
9. Yu. N. Denisyuk, Photographic reconstruction of the optical properties of an object in its
own scattered radiation field, Dokl. Akad. Nauk SSSR, v. 1444, 1275-1279,
10. 1.W. Goodman, Introduction to Fourier Optics, Second edition, McGraw-HiIl, 1988
11. E.C. Titchmarsh, Introduction to the Theoty of Fourier Integrals, Clarendon Press,
Oxford, 1948
12. L. Mertz, Transformation in Optics, Wiley, N.Y., 1965
13. G. L. Rogers, Noncoherent Optical Processing, Wiley, N.Y., 1977
14. A Lohmann, J. Opt. Soc. Am., vol. 55,1555-1556,1965
15. AS. Marathay, J. Opt. Soc. Am., vol. 4,1861-1868
16. R. V. L. Hartley, "A more symmetrical Fourier analysis applied to transmission
problems," Proc./RE, v. 30, 144-150(1942)
17. R. Bracewell, Two-dimensional Imaging, Prentice Hall, 1995
18. 1. Bertrand, P. Bertrand, 1.-Ph. Ovarlez, The Mellin Transform, in: The Transforms and
Applications Handbook, Ed. AD. Poularikas, CRC Press, 1966
19. Janke, Emde, Loesh, Tafeln Hoeherer Functionen, 6 Aufgabe, B. G. Teubner
Verlaggesellschaft, Stuttgart, 1960 ((see also http://functions.wolframcom)
20. P. Baues, Optoelectronics, v. 1,37 (1969)
21. S. A Collins, J. Opt. Soc. Am., vol. 60, p. 1168 (1970)
22. S. Abe, 1.T. Sheridan, Opt. Commun., vol. 113, p. 385 (1995)
23. V. Namias, The fractional order Foutier transform and its appIications in quantum
mechanics,1. lnst. Maths. Applics., v. 25, pp. 241-265,1980
24. A. Lohmann, Fourier curious, in: International trends in Optics, vol. 2, ICO series, 1. C,
Dainty, ed., 1994
25. A. Lohmann, Image rotation, Wigner rotation and the fractional Fourier transform, J.
Opt. Soc. Am., vol. AIO, pp. 2181-2186 (1993)
26. P. Pellat-Finet, G. Bonnet, Fractional order Fourier transform and Fourier optics, Opt.
Comm., 111, 141 (1994)
27. H. M. Ozaktas, Z. Zalevsky, M. A. Kutay, The Fractional Fourier Transform, with
application in Optics and Signal Processing, Wiley, N. Y., 2000
28. CRC Handbook of Mathematical Sciences, 6 th Edition, W. H. Beyer, Editor, CRC Press,
Roca Baton, 1978
78 Chapter2

29. H. Barrett, The Radon Transform and its Applications, In: Progress in Optics, ed. By E.
Wolf, Elsevier, Amsterdam, vol. 21, pp. 219-286, 1984,.
30. S. R. Deans, Radon and Abel Transforms, in: The Transforms and Applications
Handbook, Ed. AD. Poularikas, CRC Press, 1966
31. Y. Meyer, Wavelets: Algorithms and Applications, SIAM, 1993
32. S. G. Mallat, A theoty for multiresolution signal decomposition: The wavelet
representation, IEEE Trans. Pattern Anal. Machine IntelI., PAMI-31, pp. 674-693,1989
33. S. Mallat, A Wavelet Tour ofSignal Processing, 2nd edition, Academic Press, 1999
34. M. VetterIi, J. Kovacevic, Wavelets and subband coding, Prentice Hall, Englewood
Cliffs, NJ., 1995
35. A Haar, Zur theorie der orthogonalen funktionensysteme, Math. Annal., Vol. 69,pp.
331-371, 1910
36. A Papoulis, Probability, Random Variables and Stochastic Processes, McGrow-Hill,
N.Y., 1984
37. J.W. Goodman. Statistical Properties of Laser Speckle Patterns, In: Laser Speckle and
related Phenomena, J. C. Dainty, Ed., Springer Verlag, Berlin, 1975
 Chapter 3

DIGITAL REPRESENTATION OF SIGNALS

3.1 PRINCIPLES OF SIGNAL DIGITIZATION

As it was discussed in Sect. 2.1, signal digitization can be treated in

general terms as determination, for each particular signal, of an index of the
equivalency cell to which the signal belongs in the signal space and signal
reconstruction can be treated as generating a representative signal of the cell
from its index. This is, for instance, what we do when we describe
everything in our life with words in speaking or writing. In this case this is
our brain that makes the job of subdividing "signal space" into the
"equivalency cells" of notions and recognizing which word (cell index)
corresponds to what we want to describe. The volume of our vocabulary is
about 105 + 10 6 words. The variety of signals we have to deal with in signal
processing and especially in optics and holography is immeasurably larger.
One can see this from a simple example of the number of different images
of, for instance, 500 x 500 pixels with 256 gray levels. This number is
256500x500. No technical device will ever be capable of storing so many
images for comparing them with input images to be digitized.
A solution of this problem of the digitization complexity is found in a
two step digitization procedure. At the first step of the digitization, called
signal discretization, continuous signals are converted into discrete ones
defined as a set (sequence) of numbers, that constitute signal discrete
representation. At the second stage, called scalar (element wise)
quantization, this set of numbers is entry wise converted into a set of
quantized numbers, which finally results in a digital signal that corresponds
to the initial continuous one. In the same way as written speech is a sequence
80 Chapter 3

of letters selected from an alphabet, digital representation of signals is a

sequence of sampies that can take one of a finite set of quantized values. In
terms of signal space, discretization can be regarded as introducing a
coordinate system in the signal space and determining signal coordinates.
Element wise quantization is then quantization of those coordinates.
Equivalence cells of the signal space are therefore approximated be
hypercubes with edges parallel to the coordinate axis.

3.2 SIGNAL DISCRETIZATION AS EXPANSION OVER

A SET OF BASIS FUNCTIONS. TYPICAL BASIS
FUNCTIONS AND CLASSIFICATION
In principle, there might be many different ways of converting
continuous signals into discrete ones represented by a set of numbers.
However, entire technological tradition and all technical devices that are
used at present for such a conversion implement a method that can
mathematically be modeled as computing coefficients that re~resent signals
in their expansion over a set of basis functions. Let ~ !d)(X)J be the set of
basis functions used for signal discretization. Then coefficients ~ k} of
signal discrete representations are computed as

Uk = fa{x}rid){x)c/x (3.2.1)
x

This equation is a mathematical formulation of operations performed by

signal sensors. Functions ~ id)(x)} describe spatial sensitivity of sensors,
that is sensor point spread function, or sensor aperture functions.
Such a discretization with discretization basis ~ !d){X)} assumes that
signal can be reconstructed from its discrete representation ~ k} with the
use of a reciprocal set ~ i') (x)} of reconstructing basis functions:

(3.2.2)

)
Functions ~!' (x)} characterize point spread functions of signal
reconstruction devices (such as, for instance, computer display), or their
reconstruction aperture functions. Approximation symbol in Eq. (3.2.2) is
used to indicate that discretization ~!d)(X)} and reconstruction ~!')(x)}
3. Digital representation 0/signals 81

basis functions are in practice not necessarily mutually orthogonal as they

should be for the perfect signal reconstruction. In addition, signal
reconstructed from its discrete representation is practically always only a
more or less accurate copy of the initial signal. Even if discretization and
reconstruction bases are orthogonal perfect reconstruction requires, in
general, infinite number of terms N in the reconstruction equation (3.2.2)
which is of cause not possible.
For different bases, the signal approximation accuracy for a given N
might be different. Obviously, the bases that provide better approximation
accuracy for a given N are preferable. Therefore the convergence speed of
the expansion (3.2.2) with the number ofterms N is the first issue one should
check when selecting a basis. Approaches to the selection ofbases optimal in
this respect are discussed in Sect. 3.2.5.
The second issue is that of basis complexity. In principle, discretization
and reconstruction basis functions Can be implemented in the signal
discretization and restoration devices (or, in digital processing, in computer
memory), correspondingly, in form oftemplates. However, practically this is
almost never possible because the number N of the required template
functions is very large (for instance for images it is of order ofmagnitude of
lOs + 10 7 ). This why these functions are usually generated (in hardware or
software) from a "mother" function rather then "prefabricated" and stored.

3.2.1 Shift (convolution) basis functions

The simplest method for generating set of functions from one "mother"
function is that oftranslation (coordinate shift):

(3.2.3)

Functions generated in this way are called shift basis funetions. Elementary
shift interval ~ is called the diseretization interval. It is usually, though
not always, the same for the entire range of signal coordinates x. Positions
{x =k~} are called sampling points (for images, raster points).
Two family of functions exemplify shift basis functions: reetangular
impulse funetions

<J>k(x)=ree{ x~~), (3.2.4)

where
82 Chapter 3

rect{~ }= { 1,
O~~ <1
(3.2.5)
0, otherwise

and sinc-functions

(3.2.6)

where

.
Slnc
{}
x =sin
--
x (3.2.7)
x

As it is clear from the definition (Eqs. (3.2.4-5», rectangle impulse basis

functions are orthogonal. For rectangle impulse basis llmctions

(3.2.8)

used as the discretization basis, signals are represented their mean values
over discretization intervals:

u k =-
1
fa{x}rect (x - kj},x) dx=-
1 (k+l)ll.x
Ja{x)1x=a(kj},x). (3.2.9)
j},x x j},x j},x kll.x

Reciprocal signal reconstruction basis in this case is composed of

functions

(3.2.10)

and signals are reconstructed with this basis in a form of piece-wise constant
functions (Figure 3-1):

(3.2.11)

Sinc- functions (sampling functions) are in a sense dual to rectangle

impulse ones. As it follows from the equation:
3. Digital representation 0/ signals 83

] rect[(j + 1/ 2Llx )Llx ]exp(- i21tfx }dx =

-0')

1/2.ix 1 ( 1tX )
fexp(- i2rifx"pj = -sinc - , (3.2.12)
-1/2.ix Llx Llx

their Fourier spectrum is a rectangle impulse function (see Figure 3-2).

Figure 3-1. Continuous sinusoidal signal and its eopy reeonstrueted after diseretization with
rectangle impulse diseretization and reeonstruetion basis funetions

L1x

., ./
--"--------
I
I
\j.. :::~ ~inc(~ I ix) : x
I I I

Fourier Transform ...,-------- --- -,.,.'

I
I
• ourier Transform
I
I
I
I -----:--;'-;~~;(;;;-;i~-2-)--1
I
I I I
\
------------------ ~
,------y----
, sinc(1tjLlx) '!
,,-.. . ____--.-...-- -.. . .
I ",
·~ --r 1

Figure 3-2. Duality ofreetangle impulse and sine basis funetions

 Functions { ~ Sine( 1t (x -:Ax))} can be used as the discretization

basis:

a. k = - 1 ""f a ( x ).SIßC (1t (x - k~X)) dx . (3.2.13)

~x _"" ~x

Using shift theorem and convolution theorem of the theory of Fourier

Transforrn, (Sect. 2.4) and Eq. (3.2.12) one can show that sinc-functions are
mutuallyorthogonal:

-
1 oof.
SIßC
(1t (x - kAx)).SIßC(1t (x -IAx))dx =
Ax _00 Ax Ax
~ _!Sine(:: )sine( 1t (x - t-1)Ax )) dx =
l/2ar

-l/2Ax
fexp(- i21tf(k -1)Ax}tJf = sinc(1t (k -/))= Ö (k -/)= {IO,k*1
'
k=1

(3.2.14)

Therefore functions {Sine( 1t (x -:Ax ))} are reconstruction functions for

this basis:

(3.2.15)

Because Fourier spectrum of functions {Sine( 1t (x -:Ax ))} is bounded

in the range [-li 2Ax,1I 2Ax] (see Eq.3.2.12), basis composed of them
generates signals that belong to the class of band limited functions. It also
follows from Eq. (3.2.15) that coefficients ~ k} of discrete representation of
band limited functions are equal to functions' sampies at equidistant points
{x = kAx}:

(3.2.16)
 3. Digital representation ofsignals 85

This is where the name "sampling functions" derives from. According with
the sampling theorem (see Section 3.3), the sampling functions are a gold
standard for the discrete representation of signals with shift basis functions.
Eqs. 3.2.11 and 3.2.15 can also be regarded as convolution conv(.,.) of
signals

(3.2.17a)

with reconstruction basis functions:

a(x) = conv(i(x),CPd (x)) =

con{%a,ö (x - kM-1'P .(x») = %a,'P.(x - kM-). (3.2.17b)

This justifies yet another name for shift basis functions, the eonvolution
basis funetions, and treatment of Eq. (3.2.17, b) for reconstruction of
continuous signals from their discrete representation as eonvolution based
interpolation.

3.2.2 Scale (multiplicative) basis functions

3.2.2.1 Sinusoidal basis functions

Scaling function coordinates is yet another method for generating basis

functions from one mother function:

(3.2.18)

Apparently, there are only a few examples of such sealing basis Junetions
also called Fourier kerneIs [1]. All of them are derivatives of sinusoidal
functions {cos(21tu / X)} and {sin(21tu / X)} . These functions are
orthogonal. Functions {cos(21tu / X)} when used as a reconstruction basis,
reconstruct signals

N-l
a{x)~ aCe) = La}e) cos{21tu / X) (3.2.19)
k=O
86 Chapter 3

that are periodical, with period X, and even ftmctions (a{x)= a{- x)).
Functions {sin(27tkx I X}}used as a reconstruction basis reconstruct signals

N-l
a(x}~ a(s) = La}s) sin(21tkx / X} (3.2.20)
k~O

that are also periodical, with period X, and odd ftmctions a(x) = -a(- x) .
Signal representation coefficients for these bases are found, respectively, as,

1 X/2
a~ = - fa(x}cos(21tkxl x)ix (3.2.21)
X -X/2

1 X/2
aZ = - fa(x}sin(21tkx I x)ix (3.2.22)
X -X/2

In reality, signals are, in general, neither even nor odd functions. There are
two ways to avoid this obstac1e in using sinusoidal basis functions. One way
is form, from signals taken at interval [0, X], auxiliary even or odd signals
defined on interval [O,2X] of double length as, respectively:

i(C)(x}={ a{x1 O:s;x<X

(3.2.23)
a(2X -x1 X:S;x<2X

and

i(S)(x}={ a(x1 O:s;x<X

(3.2.24)
-a(2X -x1 X:S;x<2X

With this signal "symmetr;zat;on", bases {cos(1tkx/ X)}and {sin(1tkxl X)}

can be used for discretization and reconstruction of arbitrary signals. Signal
representation coefficients should be computed in this case as, respectively,

_( ) 1 2X_ 1 x
Ci/ =- fii(x)cos(1tkxl x)ix=- fa(x}cos(1tkxl x)ix (3.2.25)
2X 0 X 0
3. Digital representation 0/signals 87

_() 1 2X_ 1 X
a k
s = - fii(x)sin(1tkx/ X)ra=- fa(x)sin(1tkx/ X)ra (3.2.26)
2X 0 X 0

and signals should be reconstructed from their discrete representations {rlc )}

and {rl s )} as, respectively,

(3.2.27)

or

(3.2.28)

The second way to avoid the above mentioned limitations of sinusoidal

basis functions {cos(21tkx/ X)}and {sin(21tkx/ X)} is to use them in
combination:

a(x)= ~l(aic) cos(21tkx I x)+ ais) sin(27tAx I x)), (3.2.29)

k=O

where the representation coefficients ~ and ~ 1c)} 1")}

are defined by Eqs.
(3.2.21) and (3.2.22).
All these methods of signal expansion over set of sinusoidal basis
functions are different versions of weIl known Fourier series expansion of a
signal in the interval of the length X. Mathematical treatment of Fourier
series expansion is much simplified if pairs of functions {cos(21tkx / X)} and
{sin(21tkx/ X)} are replaced by a complex exponential basis functions
{exp(i21tkx / X)} :

a(x)= La k exp(i21tkxl X) (3.2.30)

k

where coefficients ~ k } are complex numbers:

1 X/2
ak =- fa(x )exp(- i21tkx / X)ra (3.2.31)
X -X/2
88 Chapter3

that can be regarded as (1/ X )-normalized sampies of signal Fourier

transform spectrum taken at frequencies {k I X} .
While in signal discrete representation the number of terms in the
expansion (3.2.31) is always finite, in classical Fourier series expansion
assumes it is infinite. Limitation of the number of terms in Fourier series
(3.2.20) is equivalent to dropping all signal components with frequencies
higher than (N -1)1 X, where N is the number of terms. This frequency
band limitation can mathematically be explained as multiplying signal

Fourier spectrum by a rectangle function rect( f ( (N -)1)1 2X) wh ich is

N-l IX
equivalent to the signal convolution with a corresponding sinc-function,
Fourier transform of the rectangle function. As a result of the convolution,
sinc-function originated oscillations may appear at signal edges (Figure 3-3).
As it was already mentioned in Sect. 2.4.1, these specific distortions in
reconstructed signals are called Gibb's effects.

::::'::::::::::::::1,.::::::::::::::::::::::::::::::::::::::::::;:::::::..::: ..... f~: ............... :........... ·······]·..............·····i: . ::::::::··:·:·:::

• • """"""""'·l·""" ",, ...... ····:····················'t
...................,~ .....................~ .....
i ,..... " ... "".~j.. ",.. ......... ...•..................
. .....................•.....................•l.................... w-l .
,., i, ..................•!....................•1 ~
...................1................... ,,i"." .,,,, j ~ j ~ i
! .......... ...... 1 ...........::....:1....:::=.: .:.:l······:····::····::·i.~:· · · · · · · · · · i.I . . . :.::;:::J.t:::::. . : . :.::.i.. ::...........::.:.:r·:···.:.:........ !.
·..J;..·v\.l i T'CI

.::::::::::::::::::,i::::::::::::::::::::l.·.·.·.·.·.·.·.·.·.·.·.·.·.·.·.·.·.·.·.·.·,1.·.·.·.·.·.·.·.·.·.·.·.·.·.· ... ~:t. . . . . . . . . L......... 'r v ..... i···· ..·.. j................· ·.. ~·
""1'" ...............~.
...................i..............""".! ............"" . . . !..
'. i .......................... '.':.......................................
, 't:"......................................... ;:",;.................
j : .. : .
.
.......... ),..... · · · i j · .. ········ .. j·.. ··+·. ·!....·..............
i
;.......!
.. ·.·.·.·.·.·.·.·.·.·..·.·.·.·.·.·.·.I,·.·.·.·.·.·.·.·.........................••...j"

.i·······......·...... ,·........:~~.::::t::::::::::::::::::t:::::::::::::::::::t:::::: i ·········:··········· ..... y. . . . . . . . +...................;.

Figure 3-3. Gibb's effect oscillations at edges of a rectangular impulse for two different, lower
(upper graph) and higher (tower graph) band limitation

It is remarkable that only frequency of Gibb's oscillations depend on

band limitation, not their magnitude. With the increase of signal's bandwidth
oscillation frequency grows while the level of the highest oscillation
overshoot remains on the level of approximately 0.21 of the impulse height
(for oscillations at edges of a rectangle impulse).
The fundamental property of sinusoidal basis functions is that, owing to
their periodicity, they generate signals that are periodical functions.
Periodical signals are dual to signals composed from shifted delta-functions
introduced in Sect.3.2.1 (Eqs. 3.2.17, a,b). This duality will be discussed in
some more details in Sect. 3.3.
 3. Digital representation 0/ signals 89

Basis of sinusoidal functions is of especial importance for digital

holography because of its direct connections with fundamental optical
transforms such as Fourier and Fresnel transforms.

3.2.2.2 Walsh functions

For basis of exponential functions {exp(ildx / X)}, generating basis

functions by scaling their argument can be treated also as generating by
means of multiplying mother function:

k
exp(i2nu / X) = TI exp(i2nx / X) (3.2.32)
1=1

There exists yet another family of orthogonal functions buHt with the same
principle, Walsh functions. Walsh functions are distinguished by the fact
that they can assume only two values; either 1, or -1. They are generated by
multiplication of clipped sinusoidal functions called the Rademacher
functions

(3.2.33)

Any two Rademacher functions are mutually orthogonal. However, the

system of functions {rad k (x)} is incomplete in the same sense as families of
functions {cos(2nu / X)} and {sin(2nu / X)} are incomplete. The Walsh
functions are actually an extension of Rademacher functions to a complete
system. These are defined as

00 Ge

walk(x)= n(radm+l(x)Y" , (3.2.34)

",=0

where k~C is the m-th digit of the so-called Gray code of number k. Gray
code digits are generated from digits {km} of binary representation of the
number

= L km 2 m;
00

k km = 0,1 (3.2.35)
m=O

according to the following rule:

(3.2.36)
90 Chapter3

where $ stands for modulo 2 addition.

Formula (3.2.34) reveals the origin of the Walsh functions. However, for
the purpose of calculating their values another representation of the Walsh
functions::

walk (~) = TI [(-IJM+l j~ = (_I)~ok~C~M+l

m=O
(3.2.37)

is useful, where

= x / X = L~m 2- m; ~O = 0;
00

~ ~m>O = 0,1 (3.2.38)

m=O

The Walsh functions are orthogonal on the interval [0, X]. Because of their
origin as a product of Rademacher functions, multiplication of two Walsh
functions results in another Walsh function with shifted an index or
argument. The shift, called dyadic shift, is determined through the bit-by-bit
modulo 2 addition offunctions' indices or, respectively, arguments:

wal k (~ )wall (~ ) = wal k$l (~ ); (3.2.39)

(3.2.40)

00

One can regard L k~c~m+l in the definition ofWalsh as a scalar product

m=O

of vectors ~~c} and {;m+l}' In this sense one can treat basis of Walsh
functions as ascale basis. Multiplicative nature of Walsh fimctions and
exponential ones justifies yet another name for these families of bases,
multiplicative bases.
Walsh function are akin to complex exponential functions
{exp(i21tkx / X)} in one more respect. This can be seen if (-1) in Eq.
(3.2.27) is replaced by exp(i1t):

(3.2.41)

and from comparison of graphs of sinusoidal and Walsh functions shown in

figure 3-4
 3. Digital representation 0/ signals 91

Founer basIs func!loos Wals/I hJlcooos

'asO o

""'"'.- --

As one can see from the figure, an important parameter that unites both
families of functions is the number of zero crossing which coincides, for
these functions, with their index k. As it was l11entioned above, for
sinusoidal functions, this parameter has an association with the notion of
frequency. For Walsh functions, this parameter obtained name of
"sequency"([2]).
Sequency wise ordering of Walsh functions is not accidental. In
principle, ordering basis functions can be arbitrary. The only condition one
should obey is matching between signal representation coefficients {a. k } and
discretization and reconstruction basis functions ~ id)(x)} and ~ (x )}.ir)
However almost always natural ways of natural ordering exist. For instance,
for shift (convolution) bases the natural ordering is according to successive
coordinate shifts. Sinusoidal basis functions are naturally ordered in
frequency of signal Fourier transform. This why sequence (number of zero
crossing) wise ordering of Walsh functions can also be regarded natural.
However, the advantage of such an ordering is more fundamental. The
most natural requirement to ordering basis functions for signal representation
is the ordering for which signal approximation error for finite number N of
terms in signal expansion over the basis monotonically decreases with the
growth of N. For Walsh functions, it happens that, for conventional analog
signals, sequency wise ordering satisfies this requirement much better than
other methods of ordering. Figure 3-5 iIIustrates this feature of Walsh
function signal spectra. Graphs on the figure show averaged squared Walsh
spectral coefficients of image rows as functions of coefficients' indices for
sequence wise (Walsh) ordering (Ieft) and for Hadamard ordering (right). In
92 Chapter 3

Hadamard ordering, digits {km} ofbinary representation ofindex k (3 .2.35)

rather than digits ~ ~c } are used for generating Walsh functions:
00

walhad k (x) = (- 1)~ok ..I;,,+, (3.2.42)

(11 .. '" ~ ... ... • ... ... .. ... ... .. .. .. ...

oe .....!.......!.... ~ .. .. .: .. ... ",' (1.1 ..... ~ ... .. ! ... ... ~ ... .. ..' .. .. ..'
I I I I
001 ..... ;- ..... ';' .... ~ ........, ...... "', 01 .....:-...... ';' ... .. ~ ... ... ., .. .. -,

'1111 ..... I'" .. .. , ...... , ...... "'" ...... -. 1)0 ...... ,. ...... , .... , ..... ""I ...... -.

(JS

(1 4
.....

_ ..
.
~

-
,
.....

.. -
.
" ..............

...............
• ,
. J
,
..
,
......

... ..
.
_,

..
I
0' ...... Ir. ... ... , ... .. ~ .. ... J ... .. _,

i ..... 1 ...... '"i ............... -. i - T i"'''' -. ... ... -.

:: t~_~~~_: ~ ~ : ~ ~ ~ ~ ~ ~
(I) .... 03 ..... -

~ l«1 ISO 200

FII'dOrl·..m

Figure 3-5. Comparison ofimage Walsh function spectra in Walsh (Jeft) and Hadamard
(right) ordering

In conclusion note that signal expansion over Walsh function basis

approximates signals with piecewise functions just as it is the case for
rectangle impulse basis functiüns.

3.2.3 Wavelets

A distinctive feature of shift (convolution) bases is that, for them, signal

representation coefficients depend on signal va lues in the vicinity of the
corresponding sampling point, and therefore they carry local information
about signals. In contrast to them, signal discrete representation für scale
(multiplicative) bases is "global": signal representation coefficients depend
 3. Digital representation 0/ signals 93

on the entire signal. Sometimes it is useful to have a combination of these

two features in the signal discrete representation. This is achieved with
wavelet basis nmctions built using a combination of shift and scaling of a
mother fimction. A most immediate example of such a combination are
Haar funetions. Haar functions are generated from from shift basis of
rectangle impulse functions {reet(x - kl1x / 11x)} and Rademacher functions
{sign(sin(2 knx / X))}
in the following way:

hark{x)=2mradm+I{x)ree{2m ; -[k]mOd2 m ). (3.2.43)

where fit is index of the most significant non-zero digit (bit) in binary
representation of k (Eq. (3.2.35)) and [k ]mod 2 m is residual from division
of k by 2 m.
The Haar functions are orthonormal on the interval [0, X]. Graphs of first
eight Haar functions are shown in Fig. 3-6.
haro(xF:::::: ..................... .

hart (x) ===========--------

har2(X)=::::::::::::::+----.. - , - - -...-....-.. ,,-..-........-.......-..·.·,-..-..1 fit =1

har3 (x) ..··................· ....··..................r::: ..
har4 (x)-i....i
hars (x) _ _----L._+--_,~-_-_--/

har6 (x).
'~:_i

har7 (x) .....................................................................L ......L.

Figure 3-6. First eight Haar functions

The figure clearly reveals the way in which Haar fimctions are built. One
can see from the figure, that Haar functions form groups in scale and that
functions within each group are generated by shift in coordinates with shift
interval coordinated with the scale. One can also see that signals
reconstructed from their discrete representation over Haar basis functions
belong to the same family of piece-wise constant functions as signals
reconstructed in the base of rectangle impulse basis functions.
94 Chapter 3

Numerical values of Haar functions may be found at each point by

expressing them, as in the case of Walsh functions, via a binary code
argument representations (3.2.35 and 38):

(3.2.44)

At present time, numerous other wavelet bases are known ([3]).

3.2.4 Discrete bases

In digital processing signals are represented in a discrete form by their

sampies obtained as a result of the discretization. For discrete signals, signal
discretization and signal reconstruction procedures that take place in signal
data compression carried out in digital computers (see Sect. 3.6) are
implemented as decimation or pruning coefficients of signal transforms, and
as, respectively, inverse transform of decimated or pruned vectors of the
coefficients. Basis vectors that form matrices of orthogonal transforms are
most usually derived from continuous basis functions described in Sect. 3.2.
In Chapt. 4 we will show the derivation of discrete orthogonal transforms
that are discrete analogs of such integral transforms as convolution, Fourier
and Fresnel Transforrns.
Main demands to discrete orthogonal transforms are basically similar to
those to continuous bases. These are good energy compaction property (low
MSE of signal reconstruction from pruned set of trans form coefficients, see
Sect. 3.2.5) and low computational complexity. They were the driving
motives in the developments of fast orthogonal transforms in 1970-1980-th
([4-6]) when most of the transforms that are now in use were invented. In
Table 3-1 most frequently used orthogonal transforms are listed and their
definitions is provided. Computational complexity of transforms listed in the
table, except that of Haar Transform, is of the order of magnitude
O(NlogN), where N is the transform dimensionality. Computational
complexity of Haar Transform is even lower: O(N). Illustrative comparison
of energy compaction properties of most frequently used orthogonal
transforms will be given in Sect. 3.2.6.
More recently, discrete wavelet transforms gained considerable
popularity because of their relatively good energy compaction properties,
low computational complexity (similar to that of Haar Transform that is the
simplest binary example of wavelets) and their multi resolution property (as
it was mentioned, wavelet signal decomposition is hybrid, local and global).

Table 3-1. Discrete orthogonal transforms

Transforrn Transform matrix N x N
3. Digital representation 0/ signals 95

Transfonn Transfonn matrix N x N

Diserete Fourier
Transforms (DFT) Direct:
I N-l
OFT = { r.; La k exp( i2rc- kr)}
...;N k=O N

Inverse:
I N-l
10FT = { r.; La k
kr)}
exp - i2rc-
(

...;N k=O N
Diserete eosine
Transforms (DCT) Direct: OCT ={.):N CO{rc (k +:: 2}r ) }
Inverse:

IOCT ={ 2 COs(rc (k + 1I2}r)}

(1 + 0 (r )).J2N N

((k + 112))}
DCT-IV
J2
DCTIV = { JNcos rc + I/2Xr
N

IOCTIV = OCTIV
Diserete Sine
Transform (DST)
OST = {_I_
.)2N
Sin (2rc (k + lXr +
2N + 1
I))}
lOST = OST
DST-IV
OSTIV = {"';N Sin( 2rc (k + 11 ~~ + 1I2)J}
lOSTIV = OSTIV
Walsh
Transform WAL = {_l_
JN (_I)Ik~:m-,rm
.. } ., n =I
~O ogz N ; {rm }
are digits of binary representation of row index r;
~~:m-l} are digits of Gray code of bit reversed digits
{kn-m-1 } of binary representation {km} of column
index; m = O,I,••• ,n -1; {k;c = k s EB ks+ 1 }; EB - modulo
2 binary addition.
Haar Transform
HAAR = {2~ (_ly·-m-, TI 0(k EB rm )1;
...; N m=O
n- m
J
n = logz N; fit - index of the most significant
nonzero digit in binary representation of row index r;
96 Chapter 3

Input signal sampIes Highest Output signal sampIes

Unit that perfonns signallow pass filtering and 2-fold decimation

of signal sampIes

Unit that perfonns signal 2-fold zooming and interpolation

Figure 3-6. Schematic diagram of discrete wavelet (multi resolution)

signal decomposition and reconstruction
3. Digital representation 0/signals 97

There are many ways to introduce and define wavelet transform as weIl
as there is unlimited number of modifications of wavelet bases. Fig. 3-6
represents one of the most simple and straightforward method, the
algorithmic one. As it is shown in the figure, signal wavelet decomposition
is made in several steps using two basic signal processing procedures:
decimation and interpolation. As a result, signal of N sampIes is decomposed
into log2 N sub-signals with descending resolution (number of sampIes)
each of which represents different sub-bands of the signal, from high to low
frequencies with the lowest level representing signal dc component The way
how low pass filtering for decimation and how signal interpolation from its
decimated copy are implemented defines type of the wavelets. This method
of signal representation is known also by names multiresolution, sub-band
representation and pyramidal coding (see, for instance, [3,7-9]).

3.2.5 Optimality of bases. Karhunen-Loeve, Hotelling and Singular

Values Decomposition Transforms

In this seetion, we address the problem of optimality of bases in terms of

the accuracy of signal approximation with a limited number of terms in their
expansion over the bases. For generality, we will assume that signals belong
to a certain family (ensemble) of signals and that the accuracy of signal
representation is evaluated on average over this family.
Let {a{x,ro)} be a set (in terms of a parameter co E n) of signals to be
represented by their approximations in a form of finite expansions:

N-l
a(x,ro) == a(x,ro) = LU k {ro)p k (x) (3.2.45)
k=O

over orthonormal basis functions ~ k (x)} such that

f<p k (x)P; (x)ctx =0 (k,/)= Olk-11. (3.2.46)

x

Let also evaluate the approximation accuracy in terms of mean squared

approximation error (MSE):

l ll
lE(rof = la(x,ro)-a(x, ro f dx = a{x,ro)- ~Uk{ro)Pk{xf dx
(3.2.47)
98 Chapter3

Optimal values of {a. k (co)} that minimize MSE can be found by equaling to
zero derivatives of Eq. (3.2.47) over {a. k (ro )} :

(3.2.48)

from which one can obtain:

Uk (ro)= fa{x,ro )P;{x)dx, (3.2.49)

x

where * stands for a complex conjugate. This means that, for basis {<P k (x)}
used as a reconstruction basis, the reciprocal discretization basis is composed
OffllOctions ~; (x)}.
From Eq. (3.2.48) it follows that minimal MSE is equal to

(3.2.50)

By an appropriate selection of bases functions ~ k (x)} one can further

minimize average MSE over the given set n of signals as defined by
parameter Ci) :

{<Pk(X)}OPI =argmin{AVn( Jla{x,ro ~2 dx _ ~l~k{ro ~2)} =

tpk (x)} X k=O

a':.Wi"{ Avn(jla(x,Ol), dx) - AVn(~~, (Oll' )}"

arg maX{AVn(~l~ k (co ~2)} =
tpk (x)} k=O

argmaX{AVn(~l
tpk (x)}
JJa{x,ro)a· {y,ro)p; {x)P
x k=O
k (Y)dxdY)} =

arg max{~l JJ AVn(a{x,ro )a * (y,ro )~; {x}P k {Y}ttdY} =

tpk(X)} k=Ox

arg max{~l JJ R {x, y}P; {x}P k {Y}ttdY}

II (3.2.51)
tpk (x)} k=O X
3. Digital representation 0/signals 99

where AVn means averaging over the signal set n. Function Ra (x, y)

Ra (x,y)= AVn(a(x,ro)z * (y,ro)) (3.2.52)

is called "correlation function" of the set of signals {a(x,ro)}. Denote

R<I>(x)= fRa(x,y)Pk(y)ty (3.2.53)

x

Then optimal bases functions {cP k (x)} are found from

{cP k(x)} = arg max{~l f R<I>(x)P; (x)ca}

~.(x)} k;OX
(3.2.54)

From Cauchy-Schwarz-Buniakowsky inequality

(3.2.55)

it follows that solution of Eq. (3.2.54) is

R<I>(x) = fRa (x, y)P k(y)ty = A.k<P k(x) (3.2.56)

x

This means that optimal bases functions are eigen functions of integral
equation Eq.(3.2.56) with kerne I defined by the correlation function (3.2.52)
of the set of signals ([ 10, 11 D. Such bases are called Karhunen-Loeve bases.
Similar reasoning can be applied to integral representation of signals. Let
signals a(x,ro) are approximated by their integral representation over self-
conjugate basis <P (x, /):

a(x,ro)== a(x,ro)= Ja (/,ro )P(x,/)t/. (3.2.57)

F

Then mean squared approximation errOT

100 Chapter 3

(3.2.58)

will be minimal if signal integral representation 0. (j ,ro ) is obtained as:

0. (j,ro )= fa(x,ro)p * (x,j}ix (3.2.59)

x

and basis 0. (x, j) satisfies integral equation

f Ra (x, Y)P(y,j)ty = A(j)P * (x,j), (3.2.60)

x

where Ra (x,y), as before, is correlation ftmction (Eq. 3.2.52) of the set of

signals a(x,ro).
An important special case is that of signals with correlation function that
depends only on the difference of its arguments:

(3.2.61)

In this case the basis optimality condition of Eq. 3.2.60 takes the form:

co
fRa (x - Y}P(y,j)ty = A(j)P * (x,j). (3.2.62)
-co

As one can see this condition is satisfied with exponential basis ftmctions

<p (x, j) = exp(i21tjx) . (3.2.63)

Two important conclusions follow from this result:

Integral Fourier transform is an MSE optimal Karhunen-Loeve

transform for signals with correlation function that depends only on
difference of its arguments;
The best, in MSE sense, approximation of such signals is their
approximation with ftmctions whose Fourier spectrum is equal to
zero outside a certain bandwidth [-F,F] (band limitedjunctions):
3. Digital representation 0/signals 101

Ja {J,ro )P{x,J)lJ .
F
ä'{x,ro)= (3.2.64)
-F

that defines minimal approximation MSE:

The discrete equivalent of the Karhunen-Loeve expansion is Hotelling

expansion or method ofprincipal components ([12]).
Let aß) = {a k }, k = O,I,••• ,N -1 is a discrete signal from an ensemble of
signals n defined by parameter ro . Find an orthogonal transform T

( 3.2.66)

( 3.2.67)

that minimizes, on average over the signal ensemble n , Euclidean distance

( 3.2.68)

between signals and their approximation

N-l }
a- m - {
- a k-- ~ar(ol)'
<Pk,r . ( 3.2.69)

By virtue ofParseval's relation, the transform is a solution ofthe equation

T= argmaXAvn[rr!Olf] =
~',rJ r=O

ar;~rXAVn{~~~a: (a: )'<Pk)<Pn,J}. ( 3.2.70)

~."J

Denote
102 Chapter3

(3.2.71)

Then transfonn optimization equation (3.2.54) becomes:

T= argmax{I:1212 Ra (k,n}Pt)<i>n,J}
~k" J ,=0 k=O 1=0
=

art~rx{~(~(<i>n~J~Ra(k,n}Pt,,)} . (3.2.72)

From Cauchy-Schwarz-Buniakowsky inequality

( 3.2.73)

that become equality iff {at = 'Abk }, where 'A is an arbitrary constant, it
follows that the transfonn optimization equation is a transfonn is reduced to
the equation

( 3.2.74)

If no ensemble averaging is required when evaluating pruned transfonn

coefficients reconstruction, Eq. 3.2.74 becomes

N-I

Lak(a)<i>k" ='A,<i>n" . (3.2.75)

k=O

Signal decomposition with a transfonn defined by basis functions that are

solutions of Eq. 3.2.75 is called singular value decomposition (SVD.
In matrix denotations, Eqs. 3.2.74 and 75 are:

(3.2.76)

and

(3.2.77)
3. Digital representation 0/ signals 103

where R" = {R,,(k,n)} = Avnla. (a') J is signal correlation matrix, a = {a t }-

vector-column of signal sampIes, (a') - complex conjugate transposed
vector (vector-row), AN = (Aß (k,r)) - a diagonal matrix. From Eqs.
(3.2.76-77) one can see that one ofdistinctive features ofHotelling and SVD
transforms is that they diagonalize signal correlation matrix:

(3.2.78)

or, respectively, matrix obtained as "outer product" 1a. {a' t J of signal

vectors:

(3.2.79)

Eqs. (3.2.55, 73-76) determine MSE optimal bases and transforms via
signal correlation function, correlation matrix or outer product. One can also
deduce from the above reasoning that each particular orthogonal transform
can be regarded to be MSE optimal for a certain c1ass of signals whose
correlation function (matrix) satisfies these equations.
The theoretical value of Karhunen-Loeve, Hotelling and SVD transforms
in signal processing is that they provide the lower bound for compact (in
terms of number of terms in signal decomposition) signal representation for
MSE criterion of signal approximation. Their practical value is, however,
quite limited because of high complexity of generating transform basis
functions and matrices and computing signal representation coefficients.

3,2.6 Two dimensional and multi dimensional bases

Two-dimensional and multi-dimensional basis functions are most

frequently (although not always) generated as separable functions of two
variables by the product of two one-dimensional ones. The underlying
reason is the easiness of hardware and software generating the basis
functions and lower computational complexity of computing signal
representation coefficients
In the case of separable basis functions, which are the product of two
one-dimensional functions, the job of evaluating of a scalar product is
reduced to performing two one-dimensional summations rather then one
two-dimensional:
104 Chapter 3

= I IadP~~;(k,I)= I
N,-IN2-1 N, - IN2 - 1

Ur •• IadP~dl)(k }P~dl)(l)=
k=O /=0 k =O /=0

I I
N, - l N 2 -1
<i> ~dl)(l) ak./<i> ~dl)(k) (3.2.80)
k =0 /=0

As one can see from Eq. 3.2.80, direct computation of 2-D sum requires
N; N; multiplicationlsummation operations while two separable 1-0
summations require N1N2tN] + N 2) operations. For large 2_D arrays,
saving is very substantial. Note that in separable two- and multi-dimensional
bases multiplicand 1-0 basis functions are not necessarily should be
identical, though most frequently they are.

DFT DFT
(real) (imaginary)

DeT DST

Walsh
. ,..
!- ~-
~ bo
• I
I-
".
I I
,a J
r ~
Haar

"
F f-"' ;e ";}' l' .I

io;:o ~ ~ ~
" .
~ 30

Figure 3-7. Examples of8x8 sets of2-D basis functions for DFT, DST, Walsh and Haar
transforrns displayed as images
 3. Digital representation 0/ signals 105

Cumsum(Spectrum).DFT:r(O.051);DCT-<:(0.037);WaISh.g(0.072);Haar-b(0.044
1~----~.------~
. ----~.~.~
--
~-. ~-.~ ..
0.9 __ . ___ t. . . . . . . . .. _ .. ' ........ - ~-::.;.: ........ • ' ..

• • ' ' '/ "'" I •

0.8 ............ ,- .. .... ...... :. /~ ~/ ~.. .. ,- ........ .. - . ..

/ >/
0.7 . • •••. ~ . .• ' , ' y ' . . .... '. . . . . . . ' .
: ./ y : : :
0.6 ......:. / " Y . .: 'D~' . • '.' • . • . • '.'

0.5 •••••••• DCT • ,• • • • • • . . •

Walsh , .
0.4 .......... _.. .. .. .. .. Haar •••.•••••

0.3
./.
. . • •:;
.
. , • .
.
• • • '.' .
,
• . . . '.' • . • • • '.'
.

0.1 · · · · · · , ' · · · · · · , · · · · · · . · · · · · · · . ·

Index 01 ordered spectral coefficients

Figure 3·8. Comparison of image DFT,DCT, Walsh and Haar spectra. Graph illustrates the
speed with which magnitude wise sorted spectral coefficients converge to the entire image
signal energy.

Figures 3-7 demonstrates examples of sets of 2-D discrete bases functions

for OFT, OCT, Walsh and Haar transforms and Figure 3-8 shows image
spectra in these bases and how rapidly spectrum pruning error converges to
zero for different bases. One can see that, for this particular image, fraction
of the coefficients that contain 90% of the signal energy is 0,051 for OFT,
0.037 for OCT, 0.072 for Walsh transform and 0.044 for Haar transform
which evidences that DCT spectrum truncation error converges to zero faster
than that for other transforms. In Sect. 3.6, we will review applications of.
signal discrete orthogonal transforms to data compression, and in Chapt. 8
we will show their use for signal denoising and deblurring.
 106 Chapter 3

3.3 SHIFT (CONVOLUTION) BASES FUNCTIONS AND

SAMPLING THEOREM

3.3.1 1-D sampling theorem

The most widely used method for signal/image discretization is

signal/image sampling by sensing them with a set of sensors that are
equidistantly placed in signal/image coordinates. The sampling is carried out
either by a mechanical or electronic signal/image scanning with a single
sensor or with an array of sensors that work in parallel. The latter is
exemplified by modem CCD and CMOS TV cameras. Remarkable enough
that image sampling was first invented by the Nature in a form of a
compound and retinal eyes (Fig. 3-9)

a) b)

Retinal eyes (R - retina; N - nerves): flatwarm (a); bevolve

molluse (b); polyehaet worm (e); Iimpet (d); abalone (e);
nautilus pinhole (f); human eye (g).
Compound eye: 400 million old fossilized trilobite eye (h).
(Images (a-f.h) are adopted from R. Dawkins ([ 13)))

h)

Figure 3-9. Different types of vision organs in the animal kingdom. In all types of eyes one
can see devices for image discretization by sampling
3. Digital representation 0/ signals 107

Signal sampling can mathematically be treated as signal representation

over shifted bases. Let <p, (x) be a point spread function of a signal/image
reconstructionldisplay device and <P d (x) be a point spread function of a
signal discretization device given in the same measurement units. Then
discrete representation of a signal a{x) over a basis generated by shifts of
the discretization device point spread function by multiple of the
discretization interval Ax is:

Cl k = Ja {x }P Ax - kiU}dx, (3.3.la)
x

and its reconstruction over basis formed by the corresponding shifts of point
spread function of the reconstruction device is

a(x) =ar(x) =LClkq>r(X - kiU). (3.3.lb)

k

Integration over x in Eq. 3.3.la and summation over k in Eq.3.3.lb are

performed within signal boundaries.
Although usually signals are finite functions, mathematical analysis of the
discretization by sampling is much simplified if one assurnes that signals are
infinite and integer index k of shift bases functions also runs in infinite
limits:

C1!J

Cl k = Ja{x)PAx-kAx)dx (3.3.2a)
-co

00

a(x)=a,(x)= LClkq>,(x-kiU) (3.3.2b)

k=-oo

Eq.(3.3.2a) means that coefficients ~ k } of signal discrete representation

over shift basis functions {q>Ax-kiU~q>r(x-kiU)} are equidistant
sampies
00

Cl k =adAkiU) = Ja dAx)5(x-kiU}dx (3.3.3)

108 Chapter 3

of signal adf (x)

00

ad/(x)= fa(~}Pd(x-~)d~ (3.3.3)

obtained from signal a(x) by its convo1ution with a point spread function of
the discretization device. A model of the discretization device that
corresponds to such a representation is shown in Fig. 3-10.

t
Clock x=k8x

Figure 3-10. Mathematical model ofthe sampling device

Signal reconstruction from its sampled representation described by Eq.

(3.3.1a) can, in its turn, be also regarded as filtering a discrete signal

00

a(x)= Ladf(k8x)8(x-k~x) (3.3.4)

k=-oo

with a filter whose impulse response is equal to point spread function <P r (x)
of the signal reconstruction device:

00

ar(x)= fa(~}Pr(x-~)~ (3.3 .5)

-00

as it is illustrated by the flow diagram in Fig. 3-11.

3. Digital representation 0/signals 109

er;
äl.. <Q

La"l(k~~(x-k6x) .. fä(~)Pr(x-~~ -----+

k llK-~
-<Q

{ad/(kiU)}
I x=k
a,(x)

Clock

Figure 3-11. Mathematical model ofthe device for restoring continuous signals from their
sampIes

Further analysis of signal sampling is much simplified in Fourier

Transfonn domain. Compute Fourier Transfonns of signals a(x) ,
a (x),ad/(/)
r and a(x)

a(j)= fa(x)exp(i21t}X)ttx = l:a /(kfu:)exp(i21tkfu:/),

k = -oo
d (3.3.7)

where

f
00

a (I) = a( x )exp(i 21tlx )th- , (3.3.9)

- 00

00

a r (I) = fa (x )exp(i 21t/x)ttt ,

r (3.3.10)
- 00

and
110 Chapter 3

k=-oo k=-oo

are Fourier spectra of signals a(x), a (x) r and Q, (x) and

00

<l> Af) = Jcp r (x )exp(i21tjX )ctx (3.3.12)

-00

and

00

<l> d (f) = Jcp d (x )exp(i21tjX)ctx (3.3.13)

-00

are, respectively, frequency responses of the signal/image

reconstructionldisplay and discretization devices.
Consider now Eq. 3.3.7. Sampies ad/(k.1x) of signal ad/(x) can be
found from its spectrum a df ( / ) as:

00

a df (k.1x) = Ja df (p )exp(- i21tk~p }ip . (3.3.14)

Then obtain:

a{f) = f 1a (p )exp(- i 21tk.1xp';Ip exp(i21tk.1xf) =

k=-IXL oo
df

(3.3.15)

By Poisson' s summation formula:

(3.3.16)

Therefore,

(3.3.17)
 3. Digital representation 0/ signals 111

which means that sampling signal adf (x) results, in Fourier domain, in
periodical replication of its spectrum with aperiod inverse to sampling
interval Llx . This phenomenon is iIIustrated in Fig. 3-12.

I
I1 11 11 11
11 11 11 11
"
11 rI
11 "
11 rI
11
I I 11 1 I I '
I I I I I I I I
" I I I I ,I
I I I I I I I I
I I I I I I I 1
I \ I \ I \ ,\
I I 1 I I I I 1
I I r \ I I I I
I I 1 1 I I I 1
I I 1 1 I I , 1
J I 1 1 1 1 J 1
I 1 I 1 I 1 I I
I \ I I I I 1 \
J .1 J I J I J I
1 1 1 1 1 I 1 I
1 I I I 1 I 1 I
I 1 J I J 1 1 I
I \ I \ I \ I \
I 'I \ I \ I \

~ 1/ ~x"'" /
Figure 3-12. Illustration ofEq. (3 .3. 17) ofperiodical replication ofsignal spectrum in signal's
sampled representation.

In this interpretation, reconstruction of signal a(x) is reduced, according

to Eq. 3.3.5, to removing, from the set of periodically replicated copies
P-d/(I + m / ~/)} of spectrum of signal adf (x) all extra replicas but the
central P-d/(/}} one by means of masking the set by frequency response
<I> r (I) of the signal/image reconstruction device:

a(J)= a,(J) ä,(J)I>,(J) = =Lt adf( f + :)}<I>,(J)=

={Llx mtoo a(1 + ::)<1> d(f + ::)}<t>, (f) . (3.3.18)

From Eq. (3.3 .18) one can easily see what type ofsignal distortions occur
in the case of signal discretization by sampling. One can also see that perfect
(distortion less) signal restoration from the result of its sampling is possible
only if signals are band-limited:

a (f) = a (I }rect(ILlx + 1/2 ) (3.3.19)

112 Chapter 3

and discretization and restoration devices act as ideallow-pass filters:

<I> d (I) oc reet(/~ + 1/2), (3.3.20)

<I> r (j) = reet(j~ + 1/2), (3.3.21)

where

I, 0 ~ x ~ 1
reet ()
x ={ . (3.3.22)
0, otherwise

Eqs. (3.3.20-21) imply that:

<P d (x)= ~sinc(nx / ~) (3.3.23)

~

<P r (x) = sinc(nx / Ax) (3.3.24)

where sinc(x) = sin x . In this ideal case

x

a(x)= IU
k=-oo
k sinc[n(x - k~)/~] (3.3.25)

provided

1 0 0 0 0
uk = - fa(x)sinc[n (x - kAx)/~]dx = fa(x)l (x - kAx}Lx = a(kAx)
Ax -00 -00

(3.3.26)

Eqs. (3.3.25 and 26) formulate the sampling theorem: band-limited signals
can exacdy reconstructed from their sampIes taken at the inter-sample
distance that is inversely proportional to the signal bandwidth.
The above analyses resulted in the sampling theorem establishes also a
link between the volume of the signal discrete representation, i.e., the
 3. Digital representation of signals 113

number of signal sampies N, and signal length X and signal spectral

bandwidth F specified by the selected discretization interval Ax:

N=X/Ax=XF (3.3.27)

Therefore signal space-bandwidth product XF is a fundamental parameter

that defines signal degrees of freedom N .

3.3.2 Sampling 2-D and multidimensional signals

For sampling 2-D signals, 2-D discretization and restoration basis

functions should be used. The simplest way to generate and to implement in
image discretization and display devices 2-D discretization and restoration
basis functions is to use separable 2-D basis functions that are formed as a
product of 1-0 function and that work separately in two dimensions when
they are used for signal/image discretization and reconstruction:

(3.3.28)

If 2-D separable basis functions are build from 1-0 shift basis functions
~~I)(XI -kAxI);<p~2)(X2 -kAx2)} and ~~I)(XI -kAxI } <p~2)(X2 -kAx 2)}
2-D signal sampling is carried out in sampling points arranged in a
rectangular (Cartesian) raster which corresponds, in the domain of 2-D
Fourier transform, to periodical replication of signal spectrum in Cartesian
co-ordinates as it is shown in Fig. 3-12. Yet another example of a periodical
replication of image spectra in Cartesian coordinates is shown in Fig. 3-13
(b).
One can see from these figures that the Cartesian periodical replication of
2-D signal spectrum does not necessarily ensures efficient spectra dense
packing, and, correspondingly, least dense arrangement of signal sampling
points for minimization of the volume of signal discrete representation. In
general, tilted co-ordinate systems selected accordingly to the shape of the
figure that encompasses signal 2-D spectrum will secure better spectra
packing (Fig. 3-14 c,d). When sampling is carried out in a tilted co-ordinate
system, sampling functions are not anymore separable functions of two
Cartesian variables.
 114 Chapter 3

Image spectra before and after sampling

Spectrum aliasing becau e of signal undersampling

Figure 3-13. 2-D separable image sampling in a rectangular raster and periodical replication
of its spectrum in a Cartesian co-ordinate system. Image at the bottom shows spectrum
aliasing (seen as dark squares around main spectral lobes) that takes place when sampling
interval is too large.
 3. Digital representation 0/ signals 115

·... lIÖX ...

I

a) b)

,'I·.·.·.·...........•. -:1 ..•................. ','" '.' '.. ',.

• I • • • • • • • • • • • • I •

"I. '.. ',. ',' ',,',.', .'1. '.' '.' ',I',. ';.1;..:,~
•; ••:. I:.•: .•: ••: ..:. "I.•: .••', .:••:. ':!t-';': "I. "••
• I . ' . ' . ' . ' I ' . ' . ' . ' . ' . ' . ' . ' . ' ....a;;• •

'."J," ',' '•• "•

... ..'...............
.' .' .' .' .' .' ., .' .' .' .' .' ...
....... .' .'.
' , ' ' • • ' • • '. • ' . • ' • • ' • • ' . ' '. I "
• ':•• ',.',. '•••.• '.' '.' '.. ',,,.
'
..'" ,"~. ',I',.'',.', ••
• ' ••

_ ...t
."1 ....
..' ~~ .... ' .. '
•• ~.',..J:
.. '
• ••.•••..•
.".~...,!~!.~ .......~...e-...a......

.' .' .' .' .' .' ,!- .

'1 • • • • • .....-
I .' .' .' .' .' .' .' .'

::::::::::~\~~~~:.:::::::::::::::::::::::::::::::::::::::::.::::.::::
;~..,;;.'.:.•..:....:•...:••..:..":'.'.:....:..•.:....:..":'.'.:.:..:.:....
., ."" " " " " ,',,',' ,',' .' .'.' ....
c) d)

Figure 3-14. An example ofan image spectrum (a), its periodical replication in rectangular
(b) and tilted (c) coordinate systems and tilted discretization raster (d)

Two practical examples of sampling in tilted co-ordinate systems are

illustrated in Figs. 3-15, 16. Fig. 3-15 illustrates that sampling in 45° -rotated
raster coordinated with the sensitivity of human vision to spatial frequencies
of different orientation (Fig. 3-15, a) is more efficient then that in Cartesian
co-ordinates. This method of sampling is regularly used in print art (Fig. 3-
15, f). Fig. 3-16 illustrates better efficiency of hexagonal raster when spectra
of signals are isotropic and may be regarded bounded by a circle. This is the
most natural assumption regarding spectra of images genera ted by optical
imaging systems. Hexagonal sampling raster is frequently used in color
displays and print art. Hexagonal arrangement of light sensitive cells can
also be found in the retina ofhuman eye (Fig. 3-16 (d».
 116 Chapter 3

b) c)
,/
"

...........................~I- --------
f.

d) e)
· ··.........
. . . . . . . . .....
...........
. . . . .. .
.... ..
· ·.....
. . . . .. ......
. . .. .. .. .
· ..........
... ........ .. . .. .
·· ............. . ..
, .............. .
··· ...............
.. . . . . . . . . . . .. .
· .......................
.... .

· · ........................... . .

· ·.. .....
. . . ................. . .

f)

Figure 3-15. 2-D sampling: a) figure in frequency domain that i1lustrates different sensitivity
of human visual system to different spatial frequencies : higher for horizontal and vertical and
lower for diagonal ones; b), c) - periodical replication ofthe figure a) in a conventional and
45°-rotated rectangle co-ordinate systems, respectively; d), e) - corresponding image
sampling rasters; f) an example of an image taken from a conventional book printing
3. Digital representation 0/ signals 117

..'
• .' • •,. .' • .• •..
''.. •• .' .' .'• • •
'I'
....i.,.....'I.'..'.'
'.
~ -- --... -
e · . ' '. ' •
-

• . ' ,. ,'

.'.........'.'.'...,.'. .
'..
" '

..
' ' '

.'.' •.' •• •• .'.' .'• •• ••

'

..,.•••1-'.'
'. ' i. '

. .
I I. • . ' . ' •• ," I

• I•. • '
••• '
,' . ' ,.
..• •• • •• •• •. ..• '
' ' '

'
b)
a)

·· · ..
... ......... . . . .. . ... .. .. .. . .
· . ....... . .. ... . . ......... . .
· · . ......... ...... . ....... .. . ..
·· ·· ..
... . .
.
.. ... . .. ..
. . ... . ...
. .
. . .. .... . ..
. ...
... .. ..
. . . ..
· ... . . .. . . . .. . . .. .
·· · ..
. .................. ....
....... ........
. .
·' .·'.... ................... ..... . .. .
'.'

c)

Figure 3-16. Image spectra replication for Cartesian (a) and hexagonal (b) sampling: the same
number of circular spectra occupy less area when they are packed in a hexagonal co-ordinate
system then in Cartesian co-ordinates; c) - hexagonal sampling raster; d) Trichromatic retinal
mosaics: hexagonal arrangement ofcones ofhuman eye retina (adopted from [14]).

Rotating and tilting discretization rasters are not the only methods for the
optimization of 2-D and multi dimensional sampling. Two other methods are
"d;scret;zat;on w;th gaps" and d;scret;zat;on w;th sub band
decompos;t;on.([ 15])
Method of the discretization with gaps is based on the use of signal
spectra symmetry. It is ilIustrated in Fig. 3-17. Let signal spectrum is
bounded as it is shown in Fig. 3-17 a). If one rotates the signal by 90° and
superimposes the rotated copy onto the initial signal, a composite signal is
obtained whose spectrum perfectly fills a square. Therefore, the composite
signal can be optimally sampled in Cartesian co-ordinates. One can see that
for all groups of four composite and initial signal sampies centered around
the rotation center, the following relationships hold:

(3 .3.29)
118 Chapter 3

from which it follows that, from four sampIes tZl' zp Z3' Z4 J of the
composite signal one sampIe, say, Z4' is redundant because it can be found
from the other three ones:

(3.3.30)

Therefore the use of signal spectrum symrnetry by means of sampling the

composite signal obtained by superimposing the signal and its 90° rotated
copy enables saving 25% of signal sampIes. Note, however, that the initial
redundancy ofsignal spectrum (Fig. 3.17, a) is 50%.

~+j
' _0
a. a,

d e
c

Figure 3-17. Discretization with gaps: a) - a figure bounding a signal spectrum (shaded); b) -
the spectrum pattern obtained by superimposing the signal rotated by 90° and periodically
extended in Cartesian co-ordinate system; c) the discretization raster corresponding to the
spectrum's pattern (b); d) - group offour initial signal sampIes around the rotation center and
directions of their translation after the rotation; e) - the corresponding group of four sampIes
ofthe initial signal plus its 90° rotated copy. Empty circIes in (c) denote redundant sampIes.
 3. Digital representation ofsignals 119

Subband I Subband 2

Subband6

Signal
spectrum 11
~~

+ -~
~-

+0+ + I..t.:~
'
~~
V-

Figure 3-18. Explanation of discretization with sub-band decomposition

Method of discretization with sub-band decomposition is illustrated in

Fig. 3-18. Area occupied by the signal spectrum is divided into sub-bands
bounded by rectangles. Signals that correspond to sub-bands can then be
optimally sampled in Cartesian rasters. For restoration of the initial signal,
one should first restore, from their corresponding sampies, sub-band signals
and then add up them. Such a procedure allows, in principle, to achieve
minimal volume of the signal discrete representation of
N = Sx I' x 2 SfiI' f 2 = Sx l' x 2 ~
'" Sjj(k)/,
l'
where Sx .' x2 and S,Jl' I:2 are areas
k
occupied, respectively, signal and its Fourier spectrum and summation over
k assumes summation over sub-bands.
 120 Chapter 3

3.3.3 Sampling artifacts: qualitative analysis

As it is stated by the sampling theorem, signal sampling should be carried

out after signal ideal low pass filtering in the bandwidth that corresponds to
the selected sampling interval. Signal reconstruction from its sampies also
requires using ideal low pass filtering in the same bandwidth. In this case
distortions because of signal sampling are minimal. Their mean squared
value is equal to the energy of signal spectral components left outside of the
chosen bandwidth. Real signal sampling and reconstruction devices are
incapable of implementing the ideal low pass filtering. This causes
additional distortions in reconstructed signals commonly called aliasing
effects.
Two sources of aliasing effects should be distinguished: inappropriate
signal pre-filtering for the sampling and inappropriate sampled signal low
pass filtering for the reconstruction. Distortions caused by the former source
are called strohe effects. They exhibit themselves as a stroboscopic reduction
of frequency of signal components outside of the chosen bandwidth. Fig. 3-
19 and 20 illustrate this phenomenon for 1-0 and 2-D signals.

-~""'''' ,/~""--'~"' ''' '''') .;""~"-- '-~, ,"",/ ~""--

. ':;,_····_. . .6 1'"\ ''-?S''''' .' ,"", ,-~--_... _;~ " " .

.. ,I \./ <v j ( / ... ,' \ .... / 'J \.... \ , , / .. I '_l \,/

.'"\ /\ {.... f\ F\ /\ t\ /\ , ,. . .,\ / \ ,I, / \ ; '... /\ /' . /\ t, 1\
I V v \.' \I '.,. 'V tv '} v Q '.' • ../ I 'v . . ..i \' \
/\/\/'/I\t\l\ 1\/') / ;\/\,-.../\/\ / \t"\;\ /\,'\ j /\ /\
, ..' ~ :;: \' v y V" t; \T 0 4 ., '.f V V c .
t· - /". ;\ /'. "> /. /~ t· ,. ,\,". 1\: /\ /\ !', l'\ ( . /\ !~ /\ C /'.
\! \
C> /'. ;\ [ \

: v 'V 'lv Q'~, Vv t'l' VC \lV\) v '';'./ 'I V\.

Signal spectra

Figure 3-19. Strobe effects in sampling l-D sinusoidal signals. Graphs on top show sinusoidal
signals of different frequencies lower and higher the frequency that corresponds to the chosen
discretization bandwidth. Image on bottom shows spectra (bright spots) of sampled sinusoidal
signals of frequencies that increase from top to bottom. One can see that frequency of
sampled sinusoids increases until it reaches the border of the bandwidth. After that the
increase ofsignal frequency causes the decrease ofthe sampled signal frequency.
3. Digital representation 0/ signals 121

Figure 3-20. Strobe effects in sampling 2-D sinusoidal signals

Distortions caused by inappropriate sampled signal low pass filtering for the
reconstruction exhibit themselves in appearance of signal high frequencies
doubles from spectrum replicas not cleaned out by the reconstruction filter.
Für sinusoidal signals, their doubles cause low frequency beatings in the
reconstructed signal that appear as müire patterns. Owing tü this reason,
these aliasing effects are called moire effects. In images, moire effects
usually appear in form of "pixelation" illustrated in Fig. 3-21. Fig. 3-22 by
Salvador Daly illustrates artistic use of the "pixelation" effect.

Figure 3-21. Image "pixelation" effects caused by inappropriate reconstruction filtering

 122 Chapter 3

Figure 3-22. Artistic use by Salvador Daly ofthe "pixelation" effect

Numerical evaluation of aliasing effects in terms of mean squared signal

reconstruction error can be carried out using Fourier Transform domain
representation of sampling and reconstruction discussed in Sects. 3.3.1 and
3.3.2. However, mean squared reconstruction error is not always an adequate
measure of the aliasing. For instance, it does not directly evaluate such
3. Digital representation 0/ signals 123

image quality feature as image readability. Fig. 3-23 illustrates importance

ofproper signal pre- and post-filtering in image sampling and reconstruction.

baqe recovery U1d. more senec'8lly, ,Ign. . . . . -I~ .ud. iaoIe tcDcnUY. " .
'OblclM fhlll ue &mOng the most rundam.· ~"1Ift' 81lOIII tbt,CDQII f'aDcIam
bC)' Ülvolve overy known ~fTom 1M . , .....,... .....,. lc8o'MI taJe-rroa dIoI
:tamining ettc IInIcturc or unrc:s.oLved sr.,:ca " ... cbc lInIaarc or,....achal AG
ent or the tinie9t motecufes. Sialt.d in 116 I _ or,. liaiaI meIe kr Swc:d ia iu. I

atproduad,.. Untorturultdr, wheos~cd

oreao buaJd. How u ,rtIaced tO/1 Ls gJ IIftCIDbellid. ~iI,,..cdco.n Js.c'
.procIKcd,.
«Jvery probIem.is dcscrilxd likc this: Oi"en oCOWI7 .. ~ iubatbcd Hk 11m: Ghft
Ua'OIt. . . . . wtaau~aed

)1, CU, be uscd 10 fum" Da csdmatciof) M.CD,beadIO rambh a.CfÖaIIe;orJ

'lf.is COrTUpted by DoiJe.doeslh~ noisepl ! lt,bCOih4AcdbrDOite.docstMa.oisepi
s. cu 'Ne amcJiorare lfte cft'ccu of thc: nob .. CIII wo . . . .~ die- cttCICU ot thc lIIlIiJ
lC>e IIdicaJ cbuJea in /? Bven ~f , urriqt. _ rarIbl cf
BOrithm for c:omputlq]rrom g'? Wbat abo .,ntb:a fof
c.
Cl_1ft -I' tit'at 'if r .....
'11 .7f~ tt \Vh.it.
ln'l Can it be usdully lncor)JOraled in our _t
These (and others) are the kinds of qu~t ""*' Je .., ...,.,. ~ por..cd iD oar
(Md ~) are dIf. tWJ o( ~
_ . :.... .s.f •..,;, ... ,. ir ,. ..... ",,,.rft""". Ift'-."; r 'h,..,.. .--. Ce ............... 1. ,_ ....................... ",t".'- t. ~

Initial image Ideally 1:2 sampled and reconstructed image

hww 11L!IXt'f~ Md, t&')O[t :tH'lH'.1Ilr. ~ I ..... , . . . feolWllJ"", ~

-OOI1eIN ~I .n; IITMX'I~ I'I-a; m". rUlu'.-n ~ 'M oIIK aiIIMIII • IDO:II: fandlm,
l1&n ID*"'.
b!JI lm"Otw .,...,. ktDwn ~.........,~ th. Mt .t.aJn ...... ~ ~f'OM, , . .
:tI:o ,UÜllru. ~ S1J"III:'.:art at IJnfICMIL"Cd. ~11'd lICHIlat.l; ehe ........ Gf uMii'A-" ~
"'" nr Ihe: lillidIL ,"n1<=~"b. &a,ed ju iLS. _ t1J6 Ale ITli i A!!II' ~. 16 •
b""'
oro'Pl:f]I ]l['IJ
.zpDdw:'ec2i!. Unfur1IJ~, 1II~5Ii'11~ 1E ..
~cdll b!~. HQ.I,I. 11 '" ullml LO J1 ~ t • oet f&lI _ .....
......,.al.Hall.,
W........'.....n
i. ~iW li IM: rM-.: (": n. m ~ , , _ FS' ....... r't::' Hkt ta.W! Q(rj..tfI
-...d
fII1IaBd lQo h ,.
71 •.;::In, hI.: ~ 111 f"m>II;oeil III li IN b:j.>rJ 11. CUI.f ttelllil!51lG rw.b1I .e:tdA.aI~oIJ
'Jl"~ lJi OXi 1ijOiIO'td~ nM, d"u unltliSll' pi , tf,tJ t:Ot.4ifC*1 ~1JCiM:. ~tfM:woi::tet pt
='. äUl J.'C ~tU rd'I1lt a.c effH IS Dr et.c: MIlt • 'fI'I m:na..e 111 IIIriNII of "" . .
.e.x r:.Iilia:.&l aJoellqtCl u. j't c.'J'\:U ~r UII~ m
I( .MIlI:!a1 d\M.rI .. .(! ~ IJ IIIIIMll
~Ihm ilJr r;;t;mp1rti"t,/hQm:' \\'fuet:>tJw:. ~hirllor""" f5 .~ """'....1 . -
ln! um 11 tI r- u~~ tTJco:por.lIIM In 00 r :AI1 ca iI bt lIIIfIIß)t iiIo.w panttd in our
llltX GIIld oc."It.I~~ LI t ,/'t: tllllt~ ... r QU:-:"I 't'haie (IM oc:a.M) II'IIIk klMf, er ~
__ :.... r •• :. .. I.: ..... ___ .... ur .~:. '1. •• 1 _ .. rwlf . . . . . . . . ~ .................... r A-:" ..... ,..

1:2 sampled and reconstructed image 1:2 sampled and reconstructed image
with no low-pass pre-filtering and with "box" pre-filtering and "box"
"box" post-filtering post-filtering

Figure 3-23. Influence of sampling artifacts on image readability: image sampling and
reconstruction with an ideal low pass filter and with a "box" filter that computes signal
average over a square box of 3x3 sampies around each pixel.
124 Chapter 3

3.3.4 Sampling artifacts: quantitative analysis

In this section, we will derive formulas that can be used for numerical
evaluation of discretization artifacts. For the sake of simplicity, I-D case will
be analyzed. For 2-D signals, one should regard signal and spectral
coordinate and indices as two-component vectors.
For discretization aperture <J> Ax), sampling interval Ax and signal
reconstruction aperture <J>r(x), signal sampies at points {(k + u)Ax}, where
u!Jx is a shift in positioning discretization aperture with respect to signal
coordinate system x, are measured as (Eq. 3.3.1a):

(3.3.31)

where

CL)

adf(x)=:: fa(~}Pd(x-~)t~ (3.3.32).

-CL)

(see Fig. 3-10) and reconstructed signal is generated as

N
ar (x)= ~:ak<J> r [x - (k + v)!Jx] (3.3.33)
k=-N

where 2N + 1 is the number of sampies used for the reconstruction and v!Jx
is shift of the reconstruction aperture with respect to the signal coordinate
system.
Find an explicit relationship between a(x) and ar (x) that involves
discretization and reconstruction device parameters:

N
ar (x)= Lak<J>r [x - (k + v)!Jx] =
k=-N
3. Digital representation 0/ signals 125

1 00 N 00

LU Ja(~ ~ LCPd [~ - (k + u).1x}p r [x - (k + V)LU]= Ja(~ )h(x,~}/~ ,

-00 k=-N -00

(3.3.34)

where

(3.3.35)

is an equivalent impulse response of the discretizationJreconstruction

procedure.
As we saw it in Sect. 3.3.1, it is easier to analyze sampling artifacts in
signal Fourier domain. Find Fourier spectrum a r (J) of the reconstructed
signal and connect it with spectrum a (J) of the initial signal:

a, (f) =}, (x )exp(i2.p).Ix = -I {}(s )h(x,s)ds } exp(i2.p).Ix =

LI {In (p )exp(- i27tp1',)dp }h(X,I', )exp(i 2xjX ).Ixdl', =

Ja(p}/p J Jh(x,~)exp[i21t(jX- p~)]exp(-i21t}/pdx~ =

00 00 00

-00 -00-00

00

Ja(p)H(J,p}/p, (3.3.36)
-00

where H(J, p) is a frequency response of the sampling/reconstruction

procedure:

00

H(J,p)= Jh(x,~ )exp[i21t (jX - p~ )]dxd~ =

-00

1
J J L cP
00 00 N
LU d [~ - (k + u)LU}P r [x - (k + V)LU ]exp[i21t (jX - p~ )]dxd~ =
-00 -00 k=-N
126 Chapter3

1 N
L J Jep AI; - (k + u)LU}p, [x - (k + v}ßx]exp[i21t (jX - pI; )]dx~ =
00 00

LU
k=-N -00-00

1
L Jep AI; - (k + u)Ax]exp(- i21tpl;)tI; x
N 00

LU
k=-N -00

00

Jep, [x - (k + v}ßx ]exp(i21tft)lx =

-00

~ f jep d (f)exp~ i21tp~- + (k + u}ßxl}df x

k=-N -00
00

Jep, (:X)exp{i21t/[:X + (k + v)LU ]}tÜ =

-00

~ fexp[- i21tp(k + u}ßx ]exp[i21t/ (k + v}ßx] x

LU k=-oo
00 00

Jep d ~-)exp(- i21tP~~f Jep, (:X)exp(i21tjX)dX =

-00 -00

where <I> d (. ) and <1>, (. ) are frequency responses of signal discretization and
reconstruction devices. Consider the summation tenn in Eq. (3.3.37):

N
L exp[i21t (I - p)kLU] =
k=-N
exp[i21t (f - pXN + 1)LU] - exp[- i21t (I - p)NLU] _
exp[i21t (I - p)Ax] - 1 -
sin(n(f - pX2N + 1}ßx] =(2N + 1) sin[1t(f - pX2N + 1)LU] .(3.3.38)
sin[1t (f - p}ßx] (2N + 1)sin(1t (I - p)LU]
Using Eq. 3.3.38, overall frequency response of the signal discretization-
reconstruction procedure can be written as

H(/,p)= exp[i21t (fv - pu)LU]. <I>~(p). <1>, (I) x

(2N + 1) sin[1t(f - pX2N + 1)LU] (3.3.39)
LU (2N + 1)sin[1t (I - p)LU]'
3. Digital representation 0/ signals 127

This equation shows that the signal discretization/reconstruction procedure

being implemented by shift basis function is nevertheless not shift invariant
for shift invariant system with PSF h(x,~) = h(x - ~) frequency response
has a form H(f, p) = H(f» (J - p). The reason of space variance is finite
number of signal sampies used for the reconstruction, or, in other words,
boundary effects. When this number increases boundary effects can be
neglected. Therefore consider asymptotic behavior of H(f, p) when
N~oo.
By Poisson' s summation formula:

~~oo k~:xP[i2nkLU(f - p)]= LUm~oo eS f - p + LU

00 00 ( m) ' (3.3.40)

obtain from Eq. 3.3.35:

Um H(f,q)= exp[i2n (fo - pU)LU]. <P~ (p). <P, (f)x

N~oo

LU m~oo eS (f -p+ ::). (3.3.41)

Substitution ofEq. 3.3.41 into Eq. 3.3.37 then gives:

Um a,(f)=
N~oo

-'!a(p)exP[i2n (fo - pu)Ax]. <I>~(p). <P,(f)m~oo eS (f - p+ .: )dP=

Cf>, (J)exp[i2n/(u - V)LU ]m~oo u( / + : : )exp( - i2n :; )- Cf> ~ ( / + ::) =

<1>, (f)<t>~ (f)x(f)exp[i2n (u - V)LU] + <P, (f)exp[i2nf(u - v)Ax]x

~[Ud! (f + : : )exp( - i2n :: ) + ad
!( f - ::) exp( i2n :: )].(3.3.43)
One can interpret Eq. 3.3.43 as one showing that the signal a, (x)
reconstructed from sampies {ak} of signal a(x) consists of 2 components.
The first component is a copy of signal a(x), shifted by (u - v)Ax and
modified by convolution with point spread functions of the discretization
128 Chapter 3

and reconstruction devices. Its spectrum is described by the first term of Eq.
3.3.41. The second component is aliasing one. Its spectrum is described by
the second term of Eq. 3.3.41. It consists of periodical replicas

{Udj{f± ;}X P(+i21t : ) } spectrum Ud/(J) ofsignal ad/(x) (Eq.

3.3.32).
In the numerical evaluation of signal distortions caused by the
discretizationJreconstruction, it is devisable to distinguish two types of
distortions that are described by the above two reconstructed signal
components: distortions owing to the signal blur by the discretization and
reconstruction apertures and aliasing distortions. The former are completely
specified by deviation of reconstructed signal spectrum from the initial one:

(3.3.44)

(note that shift factor exp[i21t (u - v}Ax] in the evaluation of this error can
be neglected as irrelevant). For the evaluation of the second type of
distortions, the most straightforward measure is that of its energy which, by
Parceval theorem, is

E;/ = JE;/ (f}df,

<X)

(3.3.45)
-<X)

where

E;/ (J)= 1<1> r (f~2 x

~F(f+ :)I'Hf+ :)1' + ~F(f -:)I},(f-:)1'

(3.3.46)

For image spectra usually decay with frequency, the value

3.3.47)

can be used as an upper bound of aliasing error.

3. Digital representation 0/ signals 129

3.4 MULTI-RESOLUTION SAMPLING

The idea of optimal sampling with sub-band decomposition described in

Sect. 3.3.2 has found its implementation in multi-resolution sampling that
gained its popularity under the name of wavelet decomposition. Multi-
resolution sampling can be most easily explained for 1-0 signals. As it is
illustrated in Fig. 3-24, signal is split into two components, low pass and
high pass ones. The low pass component has bandwidth half of that of the
initial signal and can be sampled with half of the sampling rate of the initial
signal. The residual high pass component' s spectrum bandwidth is also half
of the initial bandwidth but its spectral components are situated in the upper
half of the spectral range. Therefore, in order to sampIe it with half of the
initial sampling rate one should shift its spectrum into the lower half of the
range.

Input signal Fourier spectrum

/
-1 -112 o 1/2
High pass
Low pass
filtering
filtering

-I -1/2 0 1/2 I
(\,. .....Demodulation... / )
V a H (/) V

/
o -1/2 o 1/2

n
-1/2 1/2

n Subsampling
~
Subsampling

~ U2 U2

Figure 3-24. Multi-resolution sampling explained for a I-D signal in Fourier domain
130 Chapter3

This is achieved by means of its "demodulation" which is implemented by

multiplying the "high pass" signal component by a a sinusoidal signal whose
frequency is equal to half of the initial signal highest frequency. In multi-
resolution sampling, this procedure is repeated recursively to "low pass"
components obtained on each previous step of the recursion as it is shown in
Fig.3-25.
Input signal Channell

....
•
..... High pass
r. Demodulation Subsampling JJ 2 f-+
filter 1
Lowpass Channel2
filter 1
....
•
..... High pass
r. Demodulation Subsampling JJ 4 ~
filter 2
Lowpass Channel3
filter 2
.... High pass .... Demodulation
f+ Subsampling JJ 8 -.
filter 3
..."...............•.......•.•..•..•..........................
Channel n

Subsampling U2 n

Figure 3-25. Multi resolution signal sampling flow diagram

On each step, bandwidths of low pass and high pass filters, and therefore
signal sampling rate are half of those on the previous step. On the last n-th
step, low pass component obtained on this stage has bandwidth equal to
1/ r -th fraction of the initial signal bandwidth and it is sampled
accordingly with 1 / 2 n -th of the rate that corresponds to this initial
bandwidth. Signal reconstruction from its multi resolution sampled
representation is also carried out component (channel) wise. Each sampled
high pass component is reconstructed individually accordingly to its
bandwidth and its spectrum is shifted correspondingly and than all
reconstructed components are added up to form the entire reconstructed
signal.
Obviously, the described multi resolution sampling does not reduce the
number of signal sampies with respect to that corresponding to the signal
bandwidth. It however enables applying different quantization to different
signal components and in this way to gain in the final volume of signal
digital representation. The method has found its application in wavelet image
coding ([9]).
 3. Digital representation 0/ signals 131

3.5 UNCONVENTIONAL DIGITAL IMAGING

METHODS

In many digital imaging devices, discretization basis functions and

restoration basis functions belong to different families of functions. While
restoration basis functions implemented in common display and printing
devices are always shift (convolution) ones, in some unconventional imaging
methods discretization basis functions are other then shift ones. This
assurnes that discrete image data collected in the discretization process in
such devices should be transformed into a set of image sampIes for image
display or printing. This process usually carried out in computers is called
image reconstruction. We will briefly review three unconventional digital
imaging methods: method of coded apertures, computed tomography and
magnetic resonance imaging.
Coded aperture imaging methods (see, for instance, [16, 17]) are
suggested for non-optical imaging such as X-ray, gamma-ray and other
nuclear radiation imaging when no optical focussing devices are available.
They are an alternative to pinhole cameras that also build images directly
without any focussing radiation. In coded aperture methods, radiation from
objects to be imaged is sensed by means of binary (transparent/opaque)
masks such as Fresnel zone plates, 2-D Walsh function masks, masks built
from binary pseudo-random sequences (M-sequences) or by corresponding
arrays of parallel sensors. Such binary masks collect radiation from half of
the mask area, that is half of the image size, and thus they offer much better
ratio of signal-to-quantum noise at sensor's output than pinhole cameras that
collect radiation energy only within the area ofthe pinhole.
In computed tomography, 2-D discretization is separable in polar
coordinate system: linear array of sensors rotates around the body to be
imaged and, for a discrete set of observation angles, sampies projections of
body slices. If hA-) is an aperture function of sensors in the array, (x, y)
are coordinates in the slice plane (See Fig.2 17), (k,l) are basis function
indices; L\~ and L\8 are projection and angle discretization intervals,
discretization ba ses functions for parallel beam computed tomography is

(3.4.1)

In magnetic resonance imaging ([18]), the object to be imaged is placed

into a strong and spatially non-homogeneous magnetic field. When an
electromagnetic excitation signal on a radio frequency is applied to the
object, the object re-emits the signal but modulates its frequency. The
intensity of the signal re-emitted by different elements of the body volume is
proportional to the density of protons in these volume elements and its
132 Chapter 3

frequency is determined by the strength of the magnetic field in their

coordinates. Because the magnetic field is spatially non-homogeneous,
frequency of the re-emitted signal carries information about the spatial
coordinates of the points. Therefore, for collecting data about distribution of
proton density within the body, re-emitted signal is sampled in frequency
domain and, therefore, the discretization bases functions are sinusoidal ones.
 3. Digital representation 0/signals 133

3.6 PRINCIPLES OF SIGNAL SCALAR

QUANTIZATION

3.6.1 Optimal homogeneous non-uniform quantization

Scalar (element-wise) quantization is the second stage of signal

digitization. It is applied to signal discrete representation obtained as the
result of signal discretization. Scalar quantization implies that a dynamic
range of a finite length is chosen in the entire range of the continuous signal
representation coefficient values and is divided into quantization intervals
(Fig. 3-24a). Any value falling within a particular interval is designated by a
number (quantization interval's index) common to all values in the interval.
In signal reconstruction, this number is replaced with a value selected as a
representative of this interval (quantized value). Arrangement of
quantization intervals and selection of representative values is govemed by
the accuracy criterion for representing the continuous signal by a digital one
by probability distribution of the quantized values.

cl miD A

amin a max a ma•

a)

: : : : : : : : J: : : : : ·: : : :1: : : : : : : : :L: : : : : :· : :~: . :::::.:::::::::r:::::::::::::...:L··:::::.::::::::L:::::::::·:..::I:::::::.::::::::l:::::::::::::::

o=j~:t=:~=:jjj=tl~
-0.5 o 0.5
b)

Figure 3-26. Non-unifonn quantization. (a)-arrangement of quantization intervals and their

representative values; (b) - probability distribution function of values to be quantized; (c) -
non-unifonn quantization by means of a nonlinear pre-distortion
134 Chapter3

Let a be value of signal's discrete representation coefficient to be

quantized, Umln and u max are the corresponding representatives of values
outside the selected dynamic range, amin and a m.. are boundaries (minimum
and maximum) of the dynamic range, a (q), a (q+l) and U (q), are,
respectively, the left and right boundaries and representative value of the
q -th quantization interval (Fig. 3-26a). One can characterize the
quantization error at the q-th interval within the dynamic range as

(3.6.1)

and truncation errors of the dynamic range limitation as

E(mln) =a _u(mln). a <a . (3.6.2)

, IRin'

E(max) =a _u(max). a >a (3.6.3)

, mal: •

The requirements for quantization accuracy are generally formulated in

terms of the constraints imposed upon the quantization errors ~ (q)} • The
most common approach to formulating these constraints assurnes that the
quantization artifacts are the same for all signal representation coefficients
and therefore all coefficients are quantized in the same way. Such
quantization is called homogeneous. For the design of homogeneous
quantizer, quantization artifacts are evaluated on average over all possible
coefficient values to be quantized. To this goal, loss functions D, (E (mln»),
Dd,(E(q»), D,(E(q») that measure losses owing to the quantization errors
within and outside the dynamic range and probability density p(a) of the
values to be quantized are introduced, and quantization quality is evaluated,
separately for quantization errors within and outside the dynamic range, as

(3.6.4)
-00

(3.6.5)

and

~ Jplrv
Ia(q+l)
D d, =
\lA. \n L(q)Lrv . a
)IJd, ~ pv."
(1) =a •• a (a) =a
mm , max , (3.6.6)
q=1 a g)
 3. Digital representation 0/signals 135

where Q is the number of quantization intervals. Separate evaluation of

dynamic range limitation errors and quantization errors is advisable owing to
the different nature of these errors: while quantization errors are limited in
the range by the quantization interval size, dynamic range limitation errors
may be substantially larger.
Eqs. 3.6.4 and 5 can be used for determining dynamic range boundaries
P-mln,a m.. } given dynamic range limitation errors D, and D r • Eq. (3.6.6)
can be used for determining the set of quantization intervals boundaries
p.(9)} and representatives ti
(9)} that minimize the number of quantization

intervals Q given average quantization error Ddr or to minimize the average

quantization error given the number of quantization intervals. Such set of
quantization interval boundaries and quantization interval representatives
determine optimal scalar quantizer.
It is intuitively clear that optimal quantization intervals are, in general,
non-uniform and are defined by two factors: by probability density p(a)
and by loss function Ddr (8). The lower is the probability density in a certain
sub-range of the dynamic range of a the smaller is contribution of the
quantization error in this sub-range into the average quantization error and
therefore the larger can be quantization intervals in this sub-range. The loss
function affects size ofthe quantization intervals similarly.
There are two approaches to the design of the optimal quantizer, a
numerical optimization approach and a compressor-expander (compander)
one. Numerical optimization approach assumes analytical or numerical
solving the optimization equation:

(3.6.7)

The optimization is much simplified if loss fimction Ddr (e (9») is an even

function: Ddr(e )=Ddr (-8) such as, for instance, a quadratic loss function

Ddr (8 ) =e 2 iSo In this case, from ~

8a\9J
Ddr = 0, it follows that optimal
boundaries of the discretization intervals should be places just halfway from
the corresponding quantized values:

(9) = 'ci
a Opl (9- 1) + ci opt
~ opt
(9»)/2 • (3.6.8)

As for the representatives, they still should be found by a numericalor, when

it is possible, by analytical optimization. For instance, for the quadratic loss
136 Chapter3

function, from a:(q) Ddr = 0 it follows, in addition to Eq. (3.6.8), that

optimal representatives are centers of mass of the probability density within
the discretization intervals:

(3.6.9)

Such a solution of the quantization optimization problem for the quadratic

loss function is called Max-Lloyd quantization. In Ref. [19] one can find
tables of tx~;J ,a~;J} for optimal mean square quantizers for normal and
Laplacian probability densities.
Compressor-expander quantization was initially suggested as a method for
hardware implementation of non-uniform optimal quantization using readily
available uniform quantizers in which signal dynamic range is split into
equal quantization intervals and centers of the intervals are used as
quantization levels. In order to implement a non-uniform quantization with
uniform quantizers, signal, before being sent to uniform quantizer is
subjected to a compressive nonlinear point wise transformation.
Correspondingly, at the signal reconstruction stage, quantized values are to
be subjected, in digital-to-analog converters, to the expanding nonlinear
transformation inverse to the compressing one. (Fig. 3-27).
Optimization of the compressor-expander quantization is achieved by an
appropriate selection of the compressive nonlinear point wise
transformation. For analytical optimization, quantization optimization
equation (3.6.7) should be reformulated in terms of the compression
transformation function. Let w(.) is a compression transformation function,
~u is interval of uniform quantization.
Then, for a particular value a to be quantized, quantization interval ~
that corresponds to uniform quantization of values of function w(a) will be
equal to (Fig. 3-27, a)

(3.6.10)

and one can obtain optimal function wla) as a solution of equation:

w(a)= argmin{Ddr =
w(,,)
"l p(a )D
"mln
dr ( t;
dwa / da
Jda} , (3.6.11)
 3. Digital representation of signals 137

aJ---------------- I
-------2\:-------
---------------- --:------------
I I
4--:------------
--------------
I I I
-t~-~------------
I I I I
f-ri-1------------
I I I I
~-J-L..-I--+-----Jl
I I ::: :

0.5 /l(q 0.5

Uniform quantizer
Compressing transformation

QuantizatlOn interval
indices
a)
r----.-----..-----.,...----.----,,".5

I ___ _
_____ 1..
Restored
---+-----L--- 0 signal
---r-----~-----~----
: :
_____ L.. _____ ~-----;-----L..----
: : values
I I I I
I I I I
I I I I
I I I I
I I I I
I I I I
I I I I
I I I I
I I I : -0.5
va ues

Expanding transformation

Conversion of quantization interval b)

indices into uniformly quantized values

Figure 3-27. Schematic diagram of the companding quantization (a) and expanding
restoration (b)
138 Chapter 3

where Ddr (.) is a quantization 10ss function formu1ated in terms of the range
of quantization errors for each particu1ar va1ue a . The solution is provided
by the Euler-Lagrange equation, which in this case is written as:

(3.6.12)

where ~ = dw(a )/ da .
The following special cases will provide an insight into how probability
distribution p(a) and the type of loss function Ddr (.) affect optimal
arrangement of quantization intervals within the quantized value dynamic
range. We will begin with threshold quantization quality criteria that assume
that quantization error is nil if it does not exceed a certain threshold. For
threshold criteria, optimal arrangement of quantization intervals does not
depend on probability distribution p(a) because, according to the criteria,
los ses owing to quantization should be kept zero for all quantization
intervals.

Example 1. Uniform threshold criterion /or Absolute value 0/ Quantization

Error:

(3.6.13)

From Eq. (3.6.13) it follows that, for monotonie functions w(a),

w = 2Ll u / Ll thr , (3.6.14)
and, therefore, uniform quantization is optimal:

w(a)- w(a mln ) _ a -amin

(3.6.15)
w(a max )- w(a mln ) - a m.. -amin

Example 2. Uniform threshold criterion 0/ Relative value 0/ Quantization

Error:

ILiI:s; Ll thr = Öthra (3.6.16)

otherwise
3. Digital representation of signals 139

In this case,

(3.6.17)

and, therefore, uniform quantization in a logarithmic scale is optimal:

w(a)- w(a mln ) In(a /amlJ

(3.6.18)
w(a .... .J- w(amlJ == In(a m.. /amIJ'

Ihis particular case is of a special interest in image processing. Signal

quantization results in piece wise constant signals. In quantized images,
boundaries between these pieces visually appear as ''fa/se contours" (see
Fig. 3-28). A natural primary requirement for image quantization is that the
number and arrangement of quantization levels should be selected so as to
secure invisibility of false contours in displayed digital images. Visibility of
patches of constant gray level depends on their contrast and of size of
patches the constant gray level. According to known Weber-Fechner's /aw,
the ratio of brightness contrast AB,hr of a stimulus with brightness
B + AB,hr on the threshold of its visibility to background brightness B is
constant for a wide range ofthe brightness:

!l.B Ihr == (5 == const . (3.6.19)

B Ihr

Ihe constant decreases with the stimulus size. Its lowest value of 1% - 3%
corresponds to stimuli of large angular size. Iherefore it follows from
Weber-Fechner's law that optimal, for digital image display, quantization of
image sampies should be uniform in the logarithmic scale. Ihe number Q of
required quantization levels for logarithmic quantization can be found from
equation

Q = w(a ma. ) - w(amlJ = In(a ma• /amiJ

~" ° Ihr
(3.6.20)

For good quality photographic, IV and computer displays, dynamic range

a m•• /a mln ofdisplayed image brightness is about 100. Using this estimation
ofthe dynamic range and taking visual sensitivity threshold equal to 2%, one
can obtain from Eq. 3.6.20 that the required number of quantization levels is
243. Similar figures are characteristic also for human audio sensibility. Ihis
was the main reason why 256 quantization levels (8 bits) were chosen as a
140 Chapter 3

standard for image, sound and other analog signal representation and why 8
bits (byte) had become a basic unit in the computer industry.

Figure 3-28. False contours in image quantization (128; 64; 32; 16; 8; 4 quantization levels)
3. Digital representation 0/signals 141

Example 3. Exponential Criterion 0/Absolute Quantization Error:

(3.6.21)

Substituting (3.6.21) into (3.6.12) and solving the resulting differential

equation, obtain

(3.6.22)

Thus, the required degree of nonlinear pre-distortion depends solelyon the

probability distribution of the quantized values. The meaning of this
relationship becomes evident from the expression:

(3.6.23)

which implies that the size of quantization intervals for the various values
of a. are inversely proportional to their probability densities raised to the
corresponding power. For the widely used mean squared quantization error
criterion ( q = 1 ),

a
J(p(a ))113 da
w(a )- w(a min ) amin
(3.6.24)
w(a max ) - w(a min ) = -aj=•• -(p-(a-))-1I3-tJ,-a

Note that an interesting special case

w'a) wh ) = afp(a )da

~ - ~ min
lama.fp(a )da ( 3.6.25)
w(a max ) - W(amin ) amin amin

takes place when q =O. Such a transformation is known as histogram

equalization and is one of popular image enhancement transformations. For
142 Chapter 3

image quantization, this transformation assumes that quantization errors are

equally important whatever they values are (see Eq. (3.6.21)) and makes
quantization intervals to be inversely proportional to the probability density
for the level to be quantized (see Eq. 3.6.23).
Sometimes, as in the case quantizing spectral coefficients of signals in
Fourier, Walsh and other bases, one may regard the quantized coefficients
of the discrete signal representation as distributed according to a truncated
Gaussian probability density distribution on the interval [a min ,a max ]:

(3.6.26)

Then, for mean squared error criterion ( P = 1 ),

w(a)- w(amin) <l> (a-O:)/.J3cr a - <l> (amin-O:)/.J3cr a

w(amax)-w(amin) = <l>(a max -O:)/.J3cr a -<l>(a min -O:)/.J3cr a
(3.6.27)

where

(3.6.28)
-co

One can find that, for this case, the reduction in the required number of
quantization levels, as compared , for the same averaged quantization error,
to the case of uniform quantization, is equal to

amaxlmin (<l>(amaxlmin 1 2)- <l>(amax/min 12)]1/2

g= (3.6.29)
tf275 rl<l>~max/min
( ~) ( ~)iJ3/2
12...; 3 - <l>~max/min 12...; 3 ~
'

where a maxi min = ~a max - amin) / cr (J. • F or large a maxi min' the gain tends to
a maxlmin /5.7 .

Example 4. Exponential Criterion 0/ Relative Quantization Error:

2q
(3.6.30)
 3. Digital representation 0/signals 143

In this case, solution ofthe Euler-Lagrange equation (3.6.12) yields:

(3.6.31)

If quantized values a are distributed uniformly in the dynamic range,

optimal compression function is

wfa)- wfa ) a l /(29+ 1) _ a l /(2'1+ I)

---:,..-'~'--'-""-~~..,.m""ln~ _ mln (3.6.32)
w'a )- wfa ) - al/(2'1+I ) _a ll (2'1+ 1) •
~ mal: ~ min mal( mJn

For the mean squared relative quantization error criterion (Eq.

(3.6.30), q = 1 ):

(3.6.33)

We will refer to such type of nonlinear transformations of Eq. (3.6.32) as to

"P-th law quantizalion":

w(a)- w(a mln ) _ a p -a~ln

(3.6.34)
w{a ma. } - w(amlJ - a~.. -a~ln

P-th law quantization proved to be very useful for quantizing Fourier and
DCT transform coefficients of images. The optimal value of the nonlinearity
index P for which standard deviation of errors in reconstructed images
because of quantization of their spectra is minimal is usually about 0.3 (see,
for instance, Fig. 3-29). Remarkable enough, this corresponds to the
optimum for the mean squared relative quantization error criterion defined
by Eq. 3.6.3 3. Another option in quantization image spectral coefficients is
inhomogeneous quantization in which different coefficients are quantized
in different way. Inhomogeneous quantization in for of zonal quantization
has found its application in image transform coding (see Sect. 3.7.2)

3.6.2 Quantization in digital holography

Quantization in digital holography has its peculiarities. First of all, as it

follows from Sect. 2.3.2 (see also Sect. 13.1), this is quantization in Fourier
 144 Chapter 3

................... _.......
10 ,." '"'' '" " .• "" " ............. ",' .............", ....." ...,.. ,,,; ... ,,,,,,,, ..... ,, ...... ,,,,,,..... !.,,,,,,..,

0.2 0.4 0.6 0.8

Nonlinearity index P

Figure 3-29. Optimization of P-th law quantization of image spectrum (33 quantization levels
ofspectrum real and imaginary parts in the dynamic range of±25 oftheir standard deviation):
Initial image and images reconstructed from uniformly (P=I) and P-th (P=0.3 and 0.1) law
quantized of spectral coefficients. Bar diagram at the bottom shows how standard deviation of
quantization error depends on the nonlinearity index P. Note that while not much difference
between images for P=O.1 and P=0.3 is visible, standard deviations of the quantization error
differ substantially for these two cases.
3. Digital representation 0/ signals 145

or Fresnel transform domains. Therefore, in quantization of holograms for

digital reconstruction of images and for recording computer generated
holograms, P-th law quantization is advisable. For digital reconstruction of
holograms, corresponding correcting expanding transformations should be
implemented in hologram preprocessing (see Sect. 8. 6.3). For recording
computer generated holograms, correcting expanding transformation should
be implemented in hologram encoding for recording on optical medium.
For encoding computer generated display holograms, yet another
quantization method, the pseudo-random diffuser one, can be employed.
The method makes use of the fact that reproduction of object wave front
amplitude is only required for computer generated display holograms.
Therefore the phase component which is not visually observed can be
selected arbitrarily so as to minimize hologram quantization artifacts.
The method is mathematically described as folIows. Let be {AL}
sampIes of the object brightness with (k,l) being sampie indices. Then
sampIes of the object wave front A k / are defined as:

Ak,/ = A k,/ exp(i8 k,/)' (3.6.35)

where ~ k,/} is an array of pseudo-random numbers taken from the range

[O,21t ]. The simplest solution is to use for ~ k,/} binary statistically
independent numbers that assurne values 0 and 1t with equal probabilities.
Such a pseudo-random phase modulation of the object wave front
redistributes wave front Fourier and Fresnel spectrum energy uniformly
(statistically) between all spectral coefficients so that all coefficients have
the same dynamic range. Fig. 3.30 illustrates this phenomenon of spectrum
"equalization" by image pseudo-random phase modulation.

Initial image Spectrum of the initial image

Figure 3-30. Image Fourier spectrum equalization by means of pseudo-random phase

modulation
146 Chap/er 3

Image pseudo-random phase modulation eliminates the need of

nonuniform or inhomogeneous quantization and simplities the hologram
encoding for recording on optical media.
The pseudo-random phase component can be further optimized by an
iterative optimization procedure aimed at reducing image reconstruction
error and similar to that described in Sect. 7.2.4. The optimization is
especially required for recording kin%rms, computer generated holograms
in which hologram amplitude is quantized to only one level (see Sect. 7.2.4).
Fig. 3-31 illustrates effectiveness of P-th law and pseudo-random diffuser
quantizations for hologram encoding.

Figure 3-31. P-th law and pseudo-random diffuser quantization for encoding computer
generated display Fourier holograms: a) , b) - simulated images reconstructed from a Fourier
hologram ofthe image ofFig. 13-30, real and imaginary parts ofwhich are, correspondingly,
uniformly and P-th law (P=0.3) quantized to 257 quantization levels (2x128 for positive and
negative values plus one level for zero value); c) image reconstructed from a Fourier
hologram ofthe same image computed with the use ofpseudo-random binary (0 and n) phase
component assigned to the image.
 3. Digital representation 0/signals 147

3.7 BASICS OF SIGNAL CODING AND DATA

COMPRESSION

3.7.1 Signal rate distortion function, entropy and statistical co ding

Purpose of signal coding and data compression is encoding signals from

a signal ensemble into a stream ofbinary digits, or bits (binary code) ofthe
least possible length as evaluated on average over all possible signals. For
continuous signals, the lower bound of the code length is given by rate
distortion function H 6 of the signal ensemble A given accuracy of signal
reconstruction from the obtained digital signals:

(3.7.1)

where A is a signal ensemble reconstructed from digital signals obtained by

the encoding process from signal ensemble A and I(A, A / D(A, A)~ E) is
mutual information between the ensembles provided error d(A, A)
of
representation of signals from ensemble A by corresponding reconstructed
signals form ensemble A does not exceed certain value E ([20]). The
meaning of this measure can be illustrated by rate distortion function of an
ensemble of signals that can be modeled as Gaussian random processes with
uniform spectral density within finite bandwidth Fand that are required to
be reconstructed with root mean squared error E :

(3.7.2)

where IAI 2
is variance of the signal ensemble. The lower bound defined by
Eq. (3.7.1) can, in principle, be achieved as a result of general digitization
process discussed in Sect. 2.1.2.
In practice, data compression is applied for digital signals obtained for
continuous signals by their discretization and element-wise quantization
discussed in this chapter. For digital signals, lower bound for the code length
is determined by the entropy of digital signal ensemble ([20]). If {An} is a
set of digital signals to be encoded, and P(A n ) is probability with which
signal A k may occur, entropy
148 Chapter3

(3.7.3)
n

of the signal ensemble defined by the signal set {A,,} and their probabilities
{P(A,,)} determines minimal number of bits per signal that are necessary to
specify any particular digital signal from the ensemble. When all signals are
equally probable, entropy, and therefore number ofbits required, is maximal
and equal to

H max (A)= IOgl (Number o[ possible signals). (3.7.4)

If, however, signal probability distribution is substantially nonuniform, the

number of required bits is much less than the upper bound defined by Eq.
3.7.4. The stream ofbits obtained in data coding according to Eq. 3.7.4 is in
this case redundant. Ratio

R = H(A)/ H ma• (A) (3.7.5)

represents a measure of the digital data redundancy. It determines the limit

for possible data compression.
If, as we assume, digital signals are represented by their quantized
representation coefficients {{a.,}n} over discretization basis functions, signal
ensemble entropy can also be computed as

H(A)= - >p({a,,})logl p({a,,}), (3.7.6)

t:l
where p({a,,}) are mutual probabilities of the sets {a,,} of coefficients for
signals {An}' In this case it is more convenient to evaluate entropy per
coefficient (e.g., per sampie, per pixel):

(3.7.7)

where N is number of the representation coefficients per signal. If

representation coefficients happen to be mutually statistically independent
such that

N-I
p({a"})= TIp(a,,), (3.7.8)
,,=0
3. Digital representation of signals 149

then

(3.7.9)

where p(a. ~q)) is probability of q-th quantization level of coefficient

an and Q is number of coefficient quantization levels. Furthermore, if
probability distribution of quantized coefficients does not depend on their
index n, entropy per coefficient will further increase to

(3.7.10)

The upper limit to the per-coefficient entropy is given by entropy of

uniformly distributed quantization levels:

(3.7.11)

In practice of digital holography and image processing as weIl as in all

other digital signal processing applications, digital data obtained as a result
of discretization and scalar quantization are very redundant because signal
representation coefficients are usually highly correlated and distributed non-
uniformly, and therefore their entropy is substantially lower then the upper
limit defined by Eq. 3.7.11.
From the above reasoning, one can see that data compression which
removes their redundancy requires, in general, assigning binary codes to
signals as a whole and that the length of the code, as it follows from Eq.
3.7.2, should be equal to binary logarithm of the inverse to its probability.
This, however is not practically feasible for the same reason why general
digitization is not practically workable and, in addition, because
determination of signal probabilities is not practically feasible. This why
binary encoding is applied, with rare exceptions, to individual quantized
signal representation coefficients. For this reason, data compression is
implemented in either two or three steps. The two step procedure:
Decorrelating signal representation coefficients. If successful, this
produces data that are less correlated but instead distributed highly
non-uniformly.
Encoding decorrelated data by allocating to each of the decorrelated
coefficients a binary code with the number of bits as c10se as possible
to binary logarithm of inverse to its probability.
150 Chapter3

is called statistical (entropy) coding. It has alsoobtained a jargon name loss-

less coding because it does not assume introducing any irreversible changes
(such as quantization) into the data and enables perfect restoration of initial
signal representation coefficients. Because decorrelating signal
representation coefficients is most frequently far to be complete, it is usually
supplemented with a secondary quantization of decorrelated data. This three
step procedure:
decorrelating signal representation coefficients
secondary quantization of decorrelated data
binary statistical encoding re-quantized data
is called lossy compression to emphasize the fact that original signal
representation coefficients can not be perfectly restored from their binary
codes because of the secondary quantization.
For more then 60 years since the first works on digital data coding,
numerous data compression methods have been developed and implemented.
They will be briefly reviewed in the next section. They, however, differ only
in the implementation of the first two steps of the above three step
compression procedure. The last one, binary statistical coding is common to
all ofthem.
Two mutually complement methods ofbinary statistical coding are known
and overwhelmingly used: variable length coding (VL-coding) and coding
of rare symbols. In data coding jargon, objects of encoding are called
symbols. VL-coding is aimed at generating, for each symbol to be coded, a
binary code word of the length as elose to logarithm of the inverse to its
probability. Originally suggested by Shannon ([20]) and Fano ([21]), it was
later improved by Huffman ([22]), and the Huffman's coding procedure had
become the preferred implementation ofVL-coding. VL-Huffman coding is
an iterative process, in which, at each iteration, to codes of two symbols with
the least probabilities bits 0 and 1, correspondingly, are added and then the
symbols are united in a new auxiliary symbol whose probability is sum of
the united symbols. The iteration process is repeated with respect to the
modified at the previous iteration set of symbols until all symbols are
encoded. If probabilities of symbols are integer power of !h, such a
procedure generates binary codes with number of bits per symbol exactly
equal to binary logarithm of the inverse to its probability, and therefore
average number of bits per symbol is equal to the entropy of the symbol
ensemble. An example of Huffman-coding is illustrated in Table 3.2 for 8
symbols (A to H).
As one can see, when one of the symbols of the ensemble has probability
that is much higher than !h while common probability of others is much less
than !h, Huffman coding is inefficient because it does not allow to allocate to
symbols less than one bit. In such cases, coding of rare symbols is applied.
3. Digital representation 0/signals 151

- A nexamp.e 0 fShFH -cod'In11;O f8 svmb0 1s

1'.a ble 32
A B C D E F G H
p(A)= P(B)= p(C)= P(D)= p(E)= P(F)= p(G)= P(H)=
Iteration 0.49 0.27 0.12 0.065 0.031 0.014 0.006 0.004
I-st - - - - - - 0 I
P(GH)=O.OI
2-nd - - - - - 0 I
p(FGH)=O.024
3-d - - - - 0 I
p(EFGH)=O.055
4-th - - - 0 I
p(DEFGH)=O.12
5-th - - 0 I
p(CDEFGH)=O.24
6-th - 0 I
P(BCDEFGH)=O.51
7-th
Binary
code
0
0 10 110 1110
I
11110 111110 I 1111110 I 1111111

Entropy H=I.9554
Average number ofbits per symbol: 1.959

There are two most popular varieties of methods for coding rare symbols:
run length coding and coding co-ordinates 0/ rare symbols. In run length
coding, it is assumed that symbols to be coded form a sequence in which
runs of the most frequent symbol occur. The encoder computes length of the
runs and generates a new sequence of symbols in which the runs of the most
frequent initial symbol are replaced with new auxiliary symbols that
designate the run lengths. This new sequence can then be subjected to VL-
coding if necessary. Run length coding is an essentially I-D procedure. For
coding 2-D data, it is applied after 2-D data are converted into I-D data by
means of a zigzag scanning. Principle of zigzag scanning is illustrated below
in Fig. 3-34.
Coding coordinates of rare symbols is an alternative to the run length
coding. In this method, positions of rare symbols (others than the most
frequent one) are found. Then a new sequence of symbols is generated from
the initial one in which all occurrences of the most frequent symbol are
removed and auxiliary symbols are added to each occurrence of rare symbols
that designate their position in the initial sequence. Coding coordinates of
rare symbols is less sensitive to errors in transmitting the binary code than
run length coding. For the former, transmission errors cause only localized
152 Chapter3

errors in the decoded symbol sequence while they result in shifts of entire
decoded symbol sequence after the erroneous one in the latter. This property
is of especial importance in image coding for transmission where
transmission channel errors, in case of run length coding, may cause
substantial deformation of object boundaries.

3.7.2 Image compression methods: a review

Digital data compression methods have been developed for and found
their major applications in audio and video data compression. In this section,
we will briefly review methods for image and video compression.
Classification diagram of image coding methods is presented in Fig. 3-32.
Coding methods are classified in it into two groups: methods in which data
decorrelation and quantization are carried out separately one after another
and methods in which one can not separate decorrelation from quantization.
The latter attempt to implement, though in a reduced form, the idea of
encoding of signals as a whole to achieve the theoretical minimum of bits
defined by Eq. 3.7.3.
In decorrelation-then-quantization methods, data decorrelation is carried
out with linear transforms. Two large families of decorrelating linear
transforms are in use: "predictive" and orthogonal transforms. In predictive
transforms, for each pixel is compared its "predicted" value is generated in a
certain way from other pixels and a prediction error is computed as a
difference between the pixel value and its "predicted" value. The prediction
error is then sent to scalar quantizer. The simplest method that implements
decorrelation by prediction is known as DPCM (from "Differential Pulse
Code Modulation). Block diagram ofDPCM coding and decoding are shown
in Fig. 3-33. DPCM assumes that images are scanned row-wise/column-wise
and, for each current pixel ak,l' prediction error is computed as (Fig. 3-34)

where iikJ is a prediction value. Prediction coefficients {c} are usually

chosen so as to minimize prediction error standard deviation as computed for
an ensemble of pixels subjected to coding. For instance, in a natural
assumption that images are statistically isotropic in all directions,

(3.7.13)

and CI and Cl are found from the equation:

3. Digital representation 0/ signals 153

Linear transforms +Element-wise q uantization:

Prcdictivc transform : - Nonuniform homogcncou

- Rccursive - Zonal
- Multircsolution - Adaptive
(image pyramid)

,
decomposition and
sub-sampling
~

Orthogonal Binary statistical coding

Transforms
Framc-wisc: - Zig-zag canning 2-0 data
-DFT - Variable length coding
- OCT - "Rare" ymbol coding
- Walsh (Run length coding; coding Coded
-Haar coordinate ofrare symbols) binary
Block-wise: stream
- OCT

Hybrid "predictive"
and orthogonal
.Al
transforms

Combioed decorreJationlquandzation:
• DPCM with feedback
- LZWcoding
- Adaptive dtscrotization
- Vector quantizaflon

Figure 3-32. Classification of digital data compression methods

154 Chapter 3

. EB ...
Statistical
~ Quantizer ~ enroder
InPU!
sca nned
si gnal
i Binary coded
Predictor
signal

i a)

..... Statistical
decoder ~
De-quantizer
~ ~
Restored signal
.
Binary coded '$l>
t
signal
P'redidor

i
b)

Figure 3-33. Block digram ofDPCM coding (a) and decoding (b)

Current
Ct pixd (k,I)

Figure 3-34. 2-D prediction for row-column image scanning method

3. Digital representation 0/ signals 155

Input image histogram; Entropy=6.2

Input image; stdev=31
0.018 ::::::::::::1:::::'::.. ':'::':::::::::::::::::"::::::::1:,,,:::,::::::1

0.014
#:f:i=,
0.006

0.002

50 100 150 200 250

2-D predicI.error histogram; Entropy=2.4

20 prediction error; stdev= 11.4

0.3
~ :

0.25 "·!'''''"·"t,,. , . . . . !'',,·'' ' ' '·i''' ' ' ' ·,,':
0.2 """""""""".""".,,,,, "" """ , """"".

0.15 ,.....: ···········t·····

· ----.......................
0.] ............:.............. '" ...... .

""'. ". 'r. ·. . . . ·!". ,,",,·,,·\

~" ~
""""l"

0.05

50 100 150 200 250

Figure 3-35. Dynamic range, standard deviation and entropy reduction by predictive
decorrelation in DPCM

Inp ut
E9
~
r- Nonuniform
scale quantizer
.... Binary
statistical
encoder
-----.

I "
sig nal

I
•
De-quaoti:rer

r
Predictor ..... .....
....
I $

Figure 3-36. Block diagram ofDPCM with feedback.

156 Chapter3

(3.7.14)

where

(3.7.15)

(3.7.16)

are correlation coefficients of pixels with their immediate neighbors in

vertical or horizontal direction, of pixels with their immediate neighbor in
diagonal direction, of pixels with their next to immediate horizontal
neighbors and of pixels that are distant one from another one sampling
interval in vertical and two intervals in horizontal directions, respectively.
Fig. 3-34 illustrates how predictive decorrelation reduces dynamic range and
entropy of signal-to-be-quantized.
DPCM with no quantization is used for loss-less image coding. With
quantization, substantially larger compression is possible although images
are reconstructed with los ses. Quantization artifacts can be reduced if
prediction error is computed from already quantized data as it is done in
DPCM withfeedback (Fig. 3-35). From classification point ofview, DPCM
with feedback can be regarded as a combined discretizationlquantization
method.
In DPCM, the prediction is practically made over only immediate vicinity
of pixels. More efficient decorrelation is achieved when larger spatial
neighborhood of pixels are involved in the prediction as it is implemented in
multi-resolution image expansion. This method, depending on the
implementation, is known under different names: pyramid coding, sub-band
decomposition coding, wavelet coding. In this method, image transform
coefficients obtained at each resolution (Fig. 3-6) are optimally quantized
and then statistically encoded. Fig. 3.36 demonstrates an example of such an
image multi-resolution decomposition.
Decorrelation with orthogonal transforms is an alternative to predictive
decorrelation. Redundancy of the image signals exhibits itself in transform
domain mostly in compaction of signal energy in small number of transform
coefficients and in contraction of the dynamic range of higher order
 3. Digital representation 0/signals 157

coefficients. While Karhunen-Loeve transfonn is the statistically the best

one in tenns of energy compaction property, it is difficult to generate and is
computationally expensive. In practice, transfonns such as DFT, DCT,
Walsh transfonns that can be computed with fast transfonn algorithms (see
Chapt. 5) are used.

Figure 3-37. Image pyramid

Two versions of transfonn coding are known: frame-wise and block-wise

ones. In frame-wise coding, image frame as a whole is transformed,
transform coefficients with very low energy are truncated and the others are
quantized. In block-wise coding, image is split into blocks and individual
blocks are transformed separately. For every block, low energy block
trans form coefficients are truncated and the others are quantized. Truncation
and quantization of transform coefficients is either predefined by a special
quantization table (zonal quanlizalion) or adaptively changed for every
block.
 158 Chapter 3

In terms of energy compaction, frame-wise transforms are, in principle,

more efficient then block-wise ones. However, block-wise transform
requires much less computations and can even be implemented in
inexpensive hardware. Moreover, it is weil suited to spatial image
inhomogeneity and allows using adaptive quantization in which way
truncating and quantizing coefficients is optimized for each individual block.
Among the block transforms, DCT had proved to be the best one and it is
put in the base of image compression standards H.261, H.262, H.263, H.320,
JPEG and MPEG ([22]). The principle of image DCT block coding is

,.=1
illustrated in Fig. 3-38.

Image split up into blocks Typical energy distribution ofDCT

I"'" I""!
coefficients ofblocks (shown as gray levels)
~

:-
~

F.s 111

r>t
I 1'- I"
I 1-, I·~

I
I
I

Table ofzonal quantizati :ü-za~ emld in,

er.; er.; er.; err I7r I7r 171..1
(V /V / / ,IV ,IV / / ,lVI,)
"V VV / / ,IV / / / / //1-1
(V ,IV ,IV ,IV ,IV / / 1'1'1,)
"V 1'/ ,IVVV I'V VI' /1'1-1
r /1/ VI/ VV ,IV 1'/ 11
VII' /1/ ,IV 1'/ I' 11
r/ /11' /11' ..IV 1'1'
VI' / / 1/1' VI' VI' VI'I' ~
/11' VI/ ,IV ,IV 1'/ 1'1' V J
... 1// VI/ VI' VI' ,lVI'
(lI' /V /V VI' ,IV VI' IV J
VII' /V VV VV ,IV VII' VII'
(V VV VV VV ,IV / / / / J
J J J "..I "..I I

Figure 3-38. Principle ofDCT block transform coding

In the image block transform coding, image is split up into blocks. DCT
of each block is then computed and spectral coefficients are quantized
3. Digital representation 0/signals 159

individually according to a quantization table of zonal quantization that

specifies the number of quantization levels allocated to each coefficients as a
function of the coefficient indices. The table is built according to typical
energy distribution of the transfonn coefficients: the higher is variance of the
coefficient, the larger number of quantization levels is allocated for
quantization of this coefficient. Obtained 2-D arrays of quantized numbers
for each block are then converted, for transmitting and storing, into I-D
sequences by zigzag scanning of the arrays.
Quantization of DCT coefficients causes certain distortions in
reconstructed images. A typical artifact caused by quantization in image
block transfonn coding is the so called blocking effect: visible
discontinuities at the borders of the blocks. These discontinuities can be
substantially mitigated if blocks are overlapping. A most known
implementation ofthis idea is lapped transfonn coding ([23]).
While in coding still images predictive or orthogonal transfonn
decorrelation are mostly used, hybrid combination of predictive and
transfonn coding has also proved its usefulness. For instance, JPEG standard
for image coding assurnes using DPCM for coding dc components of image
blocks and in video coding intra-frame redundancy is removed with block
DCT coding whereas for removing inter-frame redundancy the ideas of
predictive decorrelations are used in a fonn of object motion compensation.
To conclude this brief review, mention other then DPCM with feedback
combined decorrelationlquantization methods. For loss-less coding, LZW
(Lempel-Ziv-Welch) coding is widely used in which "a code book" is
constructed for individual symbols as weil as their combinations in the
transmitted sequence of symbols. Adaptive discretization assumes taking
sampies of the signal only if difference in the signal values from the previous
sampie exceeds a quantization interval. For images, such coding results in
generating an image "contour map". Vector quantization is an exact
implementation of the general coding signals as a whole. However, owing to
computer memory and computational capacity limitations it is applied to
groups of signal/image sampies small enough to match these limitations.
 160 Chapter3

References

1. E.C. Titchmarsh, Introduction to the theory ofFourier integrals, Oxford, New York, 1948
2. H. F. Harrnuth, Transmission ofInformation by Orthogonal Functions, Springer Verlag,
Berlin, 1970
3. S. Mallat, A Wavelet Tour ofSignal Processing, 2 nd edition, Academic Press, 1999
4. N. Ahmed, K.R. Rao, Orthogonal Transforms for Digital Signal Processing, Springer
Verlag, Berlin, Heidelberg,New Y ork, 1975
5. L. Yaroslavsky, Digital Picture Processing, An Introduction, Springer Verlag, Berlin,
Heidelberg, New York, Tokyo, 1985
6. K. G. Beauchamp, Transforms for Engineers, A Guide to Signal Processing, Clarendon
Press,Oxford,1987.
7. O.J. Burt, E.A Adelson, The Laplacian Pyramid as a compact Image Code, IEEE Trans. on
Com., Vol. 31, No. 4, pp. 5320540, Apr., 1983
8. Y. Meyer, Wavelets, Aigorithms and Applications, SIAM, Philadelphia, 1993
9. M. Vetterli, J. Kovaeevic, Wavelets and Subband Coding, Prentice Hall PTR, Englewwod
Cliffs, New Jersey, 199
10. H. Karhunen, Über Lineare Methoden in der Wahscheinlichkeitsrechnung", Ann. Acad.
Science Finn., Sero A.I.37, Helsinki, 1947
11. M. Loeve, Fonctions Aleatoires de Seconde Ordre, in.: P. Levy, Processes Stochastiques
et Movement Brownien, Paris, France: Hermann, 1948
12. H. Hottelling, Analysis of a Complex of Statistical Variables into Principal Components,
J. Educ. Psychology 24 (1933), 417:441, 498-520
13. R. Dawkins, Climbing Mount Unprobable, W.W.Norton Co, N.Y.,1998
14. H. Hofer, D. R. Willams, The Eye's Mechanisms for Autocalibration, Optics & Photonics
News, Jan. 2002
15. L. Yaroslavsky, Digital Picture Processing, An Introduction, Springer Verlag, Berlin,
Heidelberg, New York, Tokyo, 1985
16. E. Caroli, J. B. Stephen, G. Di Cocco, L. Natalucci, and A Spizzichino, "Coded Aperture
Imaging in X- and Gamma-ray astronomy", Space Science Reviews, vol. 45, 1987, pp.
349-403
17. G. K. Skinner, "Imaging with coded aperture masks", Nuclear Instruments and Methods
in Physics Research, vol. 221,1984, pp. 33-40
18. W.S. Hinshaw, AH. Lent, An introduction to NMR imaging, Proc. IEEE, Special issue on
Computerized Tomography, 71, No. 3, March 1983
19. A K. Jain, Fundamentals of Digital Image Processing, Prentice Hall, Englewood Cliffs,
1989
20. C. Shannon, A Mathematical Theory of Communication, Bell System Techn. J., vol. 27,
No. 3, 379-423; No. 4, 623-656,194
21. R.M. Fano, Technical Report No. 65, The Research Laboratory of Electronics, MIT,
March 17, 1949
22. D. A. Huffman, A Method for the Construction of Minimum Redundancy Codes, Prioc.
IRE, vol. 40, No. 10, pp. 1098-1101
22. R. Gonzalez, R. E. Woods, Digital Image Processing, Prentice Hall, Upper Saddle River,
N.J., 2002
23. H. S. Malvar, Signal Processing with Lapped Transfonns, Artech House,
Norwood, MA, 1992
 Chapter 4

DIGITAL REPRESENTATION OF SIGNAL

TRANSFORMATIONS

4.1 THE PRINCIPLES

Two principles lie in the base of digital representation of continuous

signal transformations:
Consistency principle with digital representation of signals
Mutual correspondence principle between continuous and discrete
transformations

The consistency principle requires that digital representation of signal

transformations should parallel that of signals. The mutual correspondence
principle between continuous and digital transformations is said to hold if
both act to trans form identical input signals into .identical output signals.
Thus, methods for representing, specifying and characterizing continuous
transforms digitally should account for procedures of digitizing continuous
signals and reconstructing continuous signals from their digital
representatives (FigA.l).
In Sect. 3.1 we admitted that signal digitization is carried out in a two
step procedure: discretization and element-wise quantization. Let signal
{a1
a(x) be represented in a digital form by quantized coefficients q ) } of its
discrete representation a ={ak} over a chosen discretization basis. In
Sect.2.2.1, two c1asses of signal transformations were specified, point-wise
non linear and linear ones. The digital representation of point-wise
 162 Chapter4

transfonnations is based on the definition of such transfonnations as

functions {F k ~!q) .J ofthe quantized representation coefficients.

PWTa = {Fk (a!q)J (4.1.1)

In general, the transfonnation fimction may depend on the coefficient index

k, and transfonnation is inhomogeneous. Most frequently, homogeneous
transformations are regarded whose function does not depend on coefficient
indices.

Digital signal Digital signal

+
Discretization
and
quantization
~
Digital
signal
transformations
... Reconstruction
of the continuous
signal
I
~
I
I

- _.... -- ----------- - -- -
I
I Equivalent continuous transformation
I"
Continuous input signal Continuous output signal

Figure 4-1. Mutual correspondence principle between continuous and digital signal
transformations

Because the coefficient values are quantized functions {Fk may be O}

most easily specified in a tabular fonn. The volume of the table depends
upon the number of the signal quantization levels. If this number is too large
and especially when numbers in computers are represented in floating point
fonnat, other ways of specifying the function may be used. The most
commonly used way is to specify point-wise transfonnation functions in the
fonn of a polynomial:

(4.1.2)

This settles the problem of digital representation of point-wise signal

transfonnations.
Representing linear transfonns in digital fonn is, similarly to signal
digitization, also split into two steps: discretization, and subsequent
 4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 163

quantization of the discrete representation. The discrete representation of

continuous linear transforms proceeds by analogy to the discrete
representation of continuous sifnals.
Let ~!d)(x)}, ~Y)(x)}, ~!d)(x)}, and ~!')(x)} be, respectively, the
discretization and reciprocal reconstruction bases for input a(~) and output
b(x) signals of a linear transformation L and let {an} and {Pk}be the
corresponding sets of signal representing coefficients on these bases:

a(~)= Lanfl~r)(~); an = fa(~),~d)(~}I~; (4.1.3)

n X

b(x)=La(~)= L'ktJ1r)(x);'k = fb(x},1d)(x}Lx. (4.1.4)

k X

Find what numerical parameters of the linear transformation specified by its

point spread function h(x,~) are needed to compute the set of
representation coefficients {Pk} of the linear transformation output signal
from those {an} of its input signal:

'k = Ib(xx
},~d)(x}Lx = ILa(l)W)(x}Lx =
x

fLLanfl~)(~)W)(x}Lx = LanfL(fI~)(~»)W)(x}Lx = (4.1.5)

x n n X

~an 1[1h{x,l},~)(l)i~ ]tJ1d)(x}Lx =~anhk.n

where

(4.1.6)

Therefore {hk,n} are coefficients that form discrete representation of the

linear transformation with PSF h(x,~) over discretization basis functions of
output signals ~: (x)} and reconstructing basis functions ~y )(x)} of input
signals. They completely specify linear transformations for the chosen sets
164 Chapter4

of basis functions. We will refer to them as the diserete point spread

funetion (DPSF) or diserete impulse response of the linear transformation.
Equation 4.1.5 is the equation of discrete linear transformation discussed in
Sect. 2.2.1. We refer to the transformation defined by Eq. 4.1.5 to as the
digital filter. In the context of digital processing of continuous signals, it
represents a discrete analog ofintegrallinear transformation ofEq. 2.2.7.
Eq. 4.1.5 can be rewritten in a vector-matrix form as

B=HA' , (4.1.7)

where B = {Pk }and is A = {an} are vector-columns of the representative

coefficients of output and input signals, superscript t denotes matrix
transposition and His matrix of coefficients {hk,n}'
In a special case, when the bases of the input and output signals are
identical, orthogonal and are buHt from the "eigen" functions of the
transformation:

(4.1.8)

matrix H degenerates into a diagonal matrix

(4.1.9)

where 6 k ,n is the Kronecker delta (Eq. 2.1.10), and the relationship of Eq.
4.1.5 between input and output representation coefficients is reduced to a
simple element-wise multiplication:

(4.1.10)

A discrete linear transform described by a diagonal matrix will be referred to

as the sealar filter. A general transformation described by Eq 4.1.5 will be
referred to as the veetor filter.
Eq. 4.1.6 provides a way to determine a discrete linear transformation
that corresponds to a particular continuous linear transformation for the
chosen bases of input and output signals of the transformation. One can also
formulate an inverse problem: given discrete transformation with DPSF
{hk,n} what will be the point spread function of the corresponding
continuous transformation? Substituting Eqs. 4.1.3 and 5 and into Eq. 4.1.4
obtain:
4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 165

b(x)= LJlk~!r}(X)= L(Lhn'kan i",!r)(x) =

k k n ~

~[~h." la~)p!')~}t~ }f)(x) =

(4.1.11)

la~{~~ h".9'l')~)t)(X)]d~
from which it follows that PSF heq (x,~) of the continuous linear
transformation equivalent, according to the mutual correspondence principle,
to the discrete transformation with DPSF {hk,n} is

hcont(x,~)= LLhk,n,,1d)(~»r)(x). (4.1.12)

k n

In conclusion note that in digital processing DPSF coefficients of linear

transformations are quantized. However, the number of quantization levels is
usually much higher then that of signals and therefore effects of DPSF
quantization will generally be disregarded except for special cases such as
that of Sect. 6.5.
As it was stated in Sect. 3.3.1, the most widely used method of signal
discretization is that of signal sampling that uses shifted, or convolution
bases. In what folIows, we derive discrete representation, for shifted bases,
of convolution integrals, Fourier, Fresnel and wavelet transforms.
 166 Chapter4

4.2 DISCRETE REPRESENTATION OF

CONVOLUTION INTEGRAL.
DIGITAL FILTERS

The convolution integral of a signal atxJ with shift invariant kernel

htxJ (point spread function) is defined as (Eqs. 2.3.3):
00

b{x)= Ja(e).(x-e}te. (4.2.1)

-00

Let signal discretization and reconstruction bases be

(4.2.2)

Then, substituting Eqs. 4.2.1 and 2 in Eq. 4.1.6, obtain:

h.,. = 1[lh{x -~)p(.)~ - nA<)t~}(4)(x -kM)Ix =

(4.2.3)
J Jh[x - e - (k - n}u],(r )(e»(d)(x}ixde =h
00 00

k- n
-00 -00

and Eq. 4.1.5 becomes

(4.2.4)

where summation is carried out over aB available signal sampies. Because

the spatial extent of the convolution kernel h(x) is usuaBy much less then
that of signals and, consequently, the number of sampies {hk }of the filter
DPSF is much less then that of signals, it is convenient to rewrite Eq. 4.2.4
in an alternative form as

Nh-l
bk = Lhnak-n (4.2.5a)
n=O

assuming that N h is the number of sampies of {hk}' and that the number of
signal sampies is, in accordance with infinite limits in the convolution
4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 167

integral, infinite. For two-dimensional signals, one can, in a similar way,

obtain the following discrete representation of2-D convolution integral:

(4.2.5b)

Signal transformation defined by Eqs. 4.2.5 is commonly referred to as

digital Jiltering and these equations are regarded as basic equations of signal
digital filtering by digital filters defined by their discrete PSF {hll } .
According to the correspondence principle between continuous and digital
signal transformations, let us find continuous and discrete impulse responses
and frequency responses of digital filters. In the derivation, we repeat that of
Eq. 4.1.12 for digital filters redefined in Eqs. 4.2.5 and take into account that
output signal of a digital filter {bk} has finite number, say Nb, of sampIes.
For a digital filter defined by its discrete PSF {h ll } and discretization and
reconstruction basis functions as given by Eq. 4.2.2 obtain that

b(x)= %Ibkfl(r)(x - kAx)= %r%lhnak-n )fI(r)(x - kAx)=

'%1 %Ihn -1a(l),(d)(l- (k - n~}ll ]fI(r)(x - kAx)=

_Ia(~{%I%lh.. fI(r)(x - kAx ),(d)(~ - (k - n~)]d~ . (4.2.6)

Therefore, PSF of the equivalent continuous filter is

(4.2.7)

It is more convenient to characterize equivalent continuous filter by its

frequency response found as Fourier Transform of its impulse response:

-00-00

Nb-IN,,-I
L L
00 00

f f hnfl(r)(x-nAx),(d)~-(k-n~]x
_00_00 k=O n=O
Nb-IN,,-I
pe)}ixde = L L hk f
00 00

exp[i211'(fx - ffl(d)(~),(')(x)x
k=O n=O -00-00
168 Chapter4

exp{i2.[r(x + kllX)- p(~ + (k - n lAx»=

[~~. eXP(j~nAx)[~~X~* - P}täxl] x
f .,(r)(x)exp(i21ffX}tx f.,(d)(~)exp(-i2np~}l~.
00 00

(4.2.8)
-00 -00

This expression contains 4 tenns:

where

Nh-l
DFR(P)= L hn exp(i2npnllX) (4.2.10)
n=O

is detennined by digital filter discrete point spread function {h n },

(4.2.11)
-00

and

(4.2.12)
-00

are frequency responses of the signal reconstruction and discretization

devices, respectively, and tenn

Nb-l r- -' \,.] exp[i2n{!-p}NbllX]-1

SV ( I,p ) = LexPL,2.,.,1 - p!"llX =
k=O exp[-,211(1- P)1X] -1 =

sin[n{!
- [1r{ - p}N)iX]
bllX ] r --'I
eXPL'''', - p
XN I\A-]
F . (4.2.13)
sm 1- p b -

that depends only on the number of output signal sampies.

4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 169

The term DFR(P) is referred to as discrete frequency response of the

digital filter. It is a periodical function in frequency domain. If signal
reconstruction and discretization devices are, as it is required by the
sampling theorem, ideal low pass filters, terms cI> (r >(f) and q, (d >(- p)
remove all but one periods of the discrete frequency response (compare with
signal reconstruction from its sampies). Because this is not the case for all
real devices, one should anticipate aliasing effects similar to those in signal
reconstruction.
Term SV(f, p) reflects the fact that digital filter defined by Eqs. 4.2.5
and obtained as a discrete representation of the convolution integral is
nevertheless not spatially invariant owing to the finite number Nb sampies
{bk} of the filter output signal involved in the reconstruction continuous
output signal b(x). When this number increases, contribution of boundary
effects into the reconstructed signal decreases and quality of the digital filter
approximation to the convolution integral improves. In the limit, when
Nb ~ 00 , the filter is spatially homogeneous:

lim SV(f,p)=
Nb .....oo

lim Nb
Nb .....oo
Sin[~[;(fp}N
Nb sm
bt:l
- p
exp[in-(f - p XN b - 1}\x] = 6(f - p)

(4.2.14)

Boundary effects in digital filtering require especial treatment of signal

borders. From Eq. 4.2.5 it follows that N b ~ Na' where Na is the number
of filter input signal. The equality takes place only if input signal sampies
{a n} are extrapolated outside signal borders at least by the number of
sampies N h of the filter discrete PSF. Methods of extrapolating and border
processing in digital filtering will be discussed in Sect. 5.1.4.
170 Chapter 4

4.3 DISCRETE REPRESENTATION OF FOURIER

INTEGRAL TRANSFORM.

4.3.1 Discrete Fourier Transforms

Transfonnation kernel of integral Fourier transfonn exp{i2.x} is shift

variant. Therefore, in the derivation of its discrete representation one should,
in distinction to the case of the convolution integral, account for possible
arbitrary shifts of signal and its spectrum sampIes positions with respect to
signal and its spectrum coordinate systems as it is shown in Fig. 4.2.

x
Continuos signal

Signal spectrum

llf f

Figure 4-2. Geometry of dispositions of signal and its Fourier spectrum sampIes

Let signal reconstructing and spectrum discretization basis functions be

{,~r)(x)= ~r)[x-(k+u~}, and {,!d)(X)= ~d)[r-(r+v~f»,
where k and r are integer indices of signal and its spectrum sampies and u
and v are, respectively, shifts, in fractions of the discretization intervals
Ax and llf, of signal and its spectrum sampies with respect to the
corresponding co-ordinate system. Then discrete representation of the
Fourier transfonn kernel is, according to Eq. 4.1 .6,
 4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 171

J Jexp(i21f{x )per )[x - (k + u),(d)[r - (r + v}\f}txdf =

00 00

hk,r =
-00 -00

J,,(d)(f}!f Jexp{i2,q(f + (r + v}\f)(x + (k + u}\x)1Jf,(r)(x}tx =

00 00

-QO -00

00

exp[i21r{k + u)(r + v}\xAf] J,,(d)(f)exp{i21ff(k + u}\x}df x

-00

00

Jexp{i21r[r + (r + v}\/Jx}qJr>(x}tx = exp[i21r{k + u)(r + v}\xAf]x

-00

00

JCl>(r)[r + (r + v}V,(d>(f)exp{i21ff(k + u}\x}df . (4.3.1a)

-00

where Cl> (r >(f) is frequency response of signal reconstruction device.

For discrete representation ofFourier Transform, only term

hk,r = exp[i21r{k + u)(r + v}\xAf] (4.3.1b)

is usually used and term

00

JCl>(r)[r + (r + v}\f,(d)(f)exp{i21ff(k + u}\x}df (4.3.1c)

-00

is disregarded, though, in principle, it also depends on sampie indices k and

r . This can be approved by the following reasoning. Spectrum discretization
kerne I ,,(d)(f) is a function compactly concentrated in an interval of about
Af around point f = o. Within this interval, frequency response of the
signal reconstruction device Cl> (r)[r + (r + v}\f] can be approximated by a
constant because it is a Fourier transform of the reconstruction device PSF
,,(r >(x) that is compactly concentrated around point x = 0 in an interval of
about Ax. Then Eq. 4.3.1 c can be approximated as

00

fCll(r)[r + (r + v}\f],(d)(f)exp[i21if(k + u)Ax}lf ~

-00
172 Chapter 4

00

f.,(d)(r)exP{i2Iif(k + u)1x}df =<D(d)((k + u)1x) (4.3.ld)

-00

where <D(d)(_) is a frequency response of the spectrum discretization kernel

.,(d)(f) that, in its turn and for the same reason of the compactness of
.,(d)(f), can also be approximated by a constant for argument values
(k + u)1x .
Let X be an interval occupied by the signal a(x). Then, assuming that
spectrum discretization is carried out, according to this interval, with
tlf = I/X, obtain that

(4.3.2)

where N is the number of signal samples within interval X , and Eq. 4.3.1 b
can be rewritten as

h-kr -- exp[ I'2ff (k + u)(r + v>] . (4.3.3)

, N

Then basic equation 4.1.5 of digital filtering takes, for the representation of
Fourier transform, the form:

(
a,u,v ) N-l [(k+u)(r+v)] .
cx: Lak exp i2ff (4.3.4)
k=O N

where superscripts (u, v) are kept to denote that signal and its Fourier
spectrum samples are assumed to be taken with displacements u and v with
repect to signal and spectrum co ordinate systems.
Discrete transformation (4.3.4) is orthogonal. In order to make it
orthonormal, introduce a normalizing multiplier 1/JN and obtain:
(u 1 ~l [ '2 (k + u)(r + v>]
a,' v) = r,:-; .t...ak exp I ff (4.3.5)
..;N k=O N

Similarly, for inverse Fourier Transform one can obtain:

4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 173

a(U,v) = _1_ ~la(u,v) exp[-

k ~N~ r
i2. (k + u)(r + V>]
N
(4.3.6)
"1"1 k=O

Eqs. 4.3.5 and 6 contain multipliers exp (i2. ';) and exp( - i2. ';) that

do not depend on sampIe indices and can be disregarded. In this way we

arrive at the following transforrns

a!U,v) = _l_{~ak eXP[i2R'k(r + v>]ll exP(i2R'

JN k=O N ~ N
ru) (4.3.7)

as discrete representation of direct and inverse integral Fourier Transforms.

A special case of these transforrns for signal and its spectrum sampIe shifts
u=O;v =0,

a, = 1
La ex
r;;;
N-l
k
{. kr)
,21&- (4.3.9).
"N k=O N

and

1 La, ex
a k = r;;;
N-l { kr)
- ;21&- (4.3.10)
"N ,=0 N

are known as direct and inverse Discrete Fourier Transforms (OFT). In

order to distinguish the general case from this special case, we will refer to
transforrns defined by Eqs. 4.3.7 and 8 as Shifted Discrete Fourier
Transforms SOFT(u,v).
The presence of shift parameters makes the SOFT to be more flexible in
simulating integral Fourier Transform then the conventional OFT. As we
will see in Chapt. 9, it enables
• performing continuous spectrum analysis;
• computing convolution and correlation with sub-pixel resolution;
• flexibly implementation sinc-interpolation of discrete data required
by the discrete sampling theorem.
174 Chapter4

In all applications, the OFT is the basic transform that can be computed
with Fast Fourier Transform algorithm (see Chapt. 6). Shifted OFTs can be,
for any shift parameters, computed via the OFT:

a!"'); k .~12";)~[ B, exr{ ik~ )}X{ik: ):

(4.3.11)

.1"'); k.~-Ik~)%[a,.x{-Ik;)]ex{-12,.:)
(4.3.12)

Two-dimensional SOFTeu, v;p,q) is defined for 2-0 signals of N 1 x N 2 ,

sampIes as, correspondingly,

and 2-D OFT as:

(4.3.15)

(4.3.16)
4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 175

4.3.2 Properties of Discrete Fourier Transforms

In this section, we will review basic properties of Discrete Fourier

Transfonns. It is instructive to compare them with the corresponding
properties ofthe integral Fourier transfonn.

Invertibility and sincd-function

Let ~:,v J be SDFT(u,v) coefficients of a discrete signal {a k },

k = 0,1,..., N -1. Compute the inverse SDFT with shift parameters (p,q)
from a subset of K spectral sampies ~:,v }:
a:/P,v/ q = _1_[~a~'v exJ - i21f rp)Jexp[- i2lf n(r +q)] =
JN r=O ~ N N
1 r.;
1
r.;
K-I{
L N-I
L (kV) [ (k + u)r]} x
ak exp i21f- exp i21f
"N r=O "N k=O N N

exp( -i21f~ }xp[ _i2lf n(r;q)]=

_1 ~ a k exp (i21f kv - nq)~ exp[i21f (k - n + u - p)r] =

Nk=o N r=O N
1 ~ ( '2 kv - n q ) exp[i2 / N] - 1
-L,ak exp 1 If =
N k=O N exp ;2 k - n + u - p / N -1
1 N ex p{ i_[(k + ~- (n + p )]} _ exp{ _ i_[(k + ~- (n + p )]}

N t;a k exp[.Ilf (k + u)-(n

N + p)] - exp[- .Ilf .L.-~'-N--'---=:...L.
(k + u)-(n + p)] x

exp{ilf K; 1 [(k + u)-(n + p)]}ex p (i21f kv ~ n q )=

N sin{1f ~ [(k + u)-(n+ p)]}

t;a k Nsin{;[(k+u)-(n+ p»
. (K -1)(k+u-n- p)] exp
exp[ Ilf
N
('21 If
kv-n q ) =
N
176 Chapter 4

{~a. ex{;_( K; / + 2.) }inCd[K;N;(k-n +u-p»} x

exp[-i~ K;1 +2q}}XP[i~K;1(U_ p)J. (4.3.17)

where

. ( f rK
sm -X )
sincd(K;N;x)= N (4.3.18)
NSin(fr ~)
is the discrete sinc-function, discrete analog of the continuous sinc-function
defined by Eq. 2.4.6. For integer x=k-n function sincd(N;N;k-nJ is
identical to Kronecker delta-function:

. ( NjNjk - n ) = sin(tr{k - n» = {t, k =n .

smcd
. ( fr
N sm k- --
N
n) 0, k "* n
(4.3.19)

Therefore, when u = p, v = q and K =N ,

N-I
auf p,vfq
n
=~ a sincd(N'" N' k -
~ n
n)= a n (4.3.20)
k=O

which confinns the invertibility of the SDFT(u,v). However, the substance

of Eq. 4.3.17 is richer. It shows that the SDFT(u,v) is invertible in a more
general sense. Provided appropriate modulation and demodulation of signal
and its ISDFT spectrum by exponential multipliers in Eq. 4.2.17, one can
generate sincd-interpolated signal copies arbitrarily shifted with respect to
the original signal sampIes by (u - P )-th fraction of the discretization
interval. This property is closely connected with the discrete sampling
theorem proved below.
Discrete sinc-function is a discrete analog of the continuous sinc-function
defined by Eq. 2.4.6. They are almost identical within interval of N sampIes
for the discrete sinc-function and its corresponding interval N I!ix for the
continuous delta-function as one see from Fig. 4-3. But outside this interval
4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 177

they differ dramatically: while sinc-function continues to decay, sincd-

function is a periodical function and the type of its periodicity depends on
whether N is odd or even number: sincd(N;N;k + gN)=(_IY(N-l).

o 0 0 • 0

......
• 0
• 0 0
0.8 --:- - - - - - - - ~ - - - - - - -~ - - - - - -- --~--_ ~-
o
--------!----- --:..-------.:.
o 0 0 0

0.6 - ..:--------! .. -----:..-------

: :: :::
0." --:---- -- --~ - - -------!--- - -- ~ -----.-~-
- -- - . ~-----.-

_-,·____ ____ ,. __ -----,. ------ - ...... ----.

. ---- . .
--,. ----_.-"1-
• , . I I I
"
~

lA i : : J.,J. : .d olL.. : .1 1t. : :, ~

o .

. .. .,.
0.6 -~ .............. -~ .............. ~- . . .. .. . . .. .. .. .. ........ .... ..: ................ ~ .......... .. .. .. ~

+- -------~ -------+- ------ --------~- ------ -~------ --~

• , . • I •

-!----- -- --:- -- -- --- + --

----- -------~- -------~----- ---~
06
0.4
02 -:. - - - - - - - -:.. - - - - --.:- - -- - - - - - - - - - - - ~- - -- -- ~ -- - -- -- - ;
.n IA~. : .AI : .aal fl... : 11 . : :1

~. 2
~~ : "~ IW ' : ""li W-
0

;- - -- - - -- -;- -- -- -- -;- - - - - -- - . - - - - -- - -;-- - -

: . ~:
- - ;-- - --- - --:
'r l

~.4 -~ - - - - - -- - :- - - -- ---:- - -- - - - -~- -- - - --- ~ -- - - - -~- - - --- - - ~

-0.6 -~ ................ -:- .... -- -- -;_ . . --- ---~-_ ...... _. -i-""" .... ~ -- - - ...... . ~
4'l A -:- .............. -:- ............ --:- ................ -:- .............. -:-.. .. ..
..
.. ..-1 ................ ~
..
-I L:~~--_10-:':-...L----::'::lll:----:O~--::lll=---L~40:----:60~

Figure 4-3. Discrete sincd- (solid line) and continuous sinc- (dotted line)
functions for N =33 (top plot) and N =34 (bottom plot)

Separability 0/2-D trans/orms

Separability of 2-D and, generally, multi-dimensional SDFTs easily seen
from their definitions (Eqs. 4.3.13 - 16) follows from the separability of 2-D
(multi-dimensional) integral Fourier transform and the separability of
sampling basis functions .
Periodicity
From Eq. 4.3 .8 it follows that, for integer number g, discrete signal ~:,v}
reconstructed by the inverse SDFT(u,v) from SDFT(u,v) spectrum of signal
{ak }is defined outside the index interval (0, N -1) as

a(U,v) =a(u,v) exp(- i2tr gv) (4.3.21)

k+gN k N
178 Chapter4

In particular, when v = 0, it is a periodical function of k :

iu,o)
k+gN
=a(u,o)
k
=iu{o)
(kJlllodN'
(4.3.22)

where (k )mod N is residual of kIN. Similar periodicity property holds for

SDFT(u,v) spectra:

a(u,v)
r+gN
=a(u,v)
r
exp(i2tr
N
gu) '. a(o,v) = a(o,v) = a(o{v)
r+ gN r (r JIIIod N
(4 3 23)
..

For the DFT, both signal and its spectrum are regarded as periodical:

a(o,o) =a(o\o) . a(o,o) =a(Ojo) (4.3.24)

k (kJmodN' r (rmodN·

Owing to the separability of the DFTs, the same holds for 2-D and multi-
dimensional DFTs for aU indices.
The periodicity property of DFTs is their main distinction from the
integral Fourier transform whose discrete representation they are. It
originates from image sampling with shift basis functions and from the fact
that the number of sampies in signal definition is finite.

Shift theorem

Shift theorem for the DFTs is completely analogous to that for integral
Fourier transform:

If ~k} SDFT(u,v) )~:,v land ~:,v} ISDFT(u,v) )~:,v l

!hen ~...J son1.,.). { a:" ex{-i2a*, r; v)} (4.3.25.)

Permutation theorem

The permutation theorem is an analog of the integral Fourier transform

scaling theorem. For the DFT, if P is a number that has no divisors in
4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 179

common with N, and Q is an integer inverse to P such that their modulo

N product (PQlmOd N =1 ,

~(Pk)mOdN} »PT ){a(Qr)modN}. (4.3.26)

As one can see from Eq. 4.3.26, multiplying signal sampie index results, in
distinction to the continuous case, not in the corresponding scaling of signal
and its spectrum but rather in permutations of signal and its spectrum
sampies provided the scaling factor and the number of signal sampies have
no divisors in common. The distinctive feature of the finite dimensional
discrete signal space in which the DFT operates is that sampie indices are
counted modulo N and no other indices are defined.

Convolution theorem
Let ~:",v" } be SDFT(ua,va) of a signal {ak} and {a:b,Vb} be SDFT(Ub,Vb)
of a signal {bk}' k,r =O,1,•••,N -1. Compute the ISDFT(u,",vc) of their
product:

u v
c ke• e = .JN
1 )~I u v u v
, ; !::ocrr ". "Pr b' b exp - ,21&
exp(- ,,21& kv [, r(k N+ U c >] =

1
r.;exp kv
( -i21&-c )N-I{
L 1
r.;exp ( i21&-1I )N-t [
ru Lanexp i21& n(r + 11 V)] X
"+J N N r=O "+J N N n=O N

ffi eX{26 r~ )~:bm ex{;26 m(r; v, >}X{-;26 r(i; uJJ} ~

kv
~ 1 3 exp( -i21&-c )N-lran exp( n v
21&-11- ) X
N Nn=o N

4.J '" exp(',21&--

{ ~lb 4.Jexp[',21& ren + m - k + U" + u
mv )~1 b b - ucl]} =
m=O N ,=0 N
 180 Chapter4

_1
JN exp{- i2ff[!!..-(V _ N
N C
-1) _ -1
2N
N
2N
(u + u _ u )~} x
abc J
La
N-I

n~O
n
N
N-1)]
n ( va + - -
exp[ i2ff-
2N
X

Lb
N-I
m~O
m exp[i2ff-
m ( Vb +N-l)]
N
- - sincd[N;N;tr{n+m-k+ua +U b -UJ]
2N
(4.3.27)

N-l
Select Va =Vb =-V c = - - - toobtain:
2N

(4.3.28)

where

N-l
bn = Lbm sincd[N;N;.(m - n + U a + Ub - u c )] (4.3.29)
m=O

is sincd-interpolated copy of signal {bk}' Eq. 4.3.27 is a discrete convolution

of signals {bk} and { bn }. On signal borders, it assurnes that the signals are
extended outside index interval [0, N -1] according to the shift theorem
(Eq. 4.3.25). In a special case of the DFT
(u a = ub = Uc = va =Vb = -v c = 0), it is a cyclic convolution:

(4.3.30)

In order to distinguish it from the general case of Eq. 4.3.28, we will refer to
Eqs. 4.3.27 and 28 as to shifted cyclic convolution

Symmetry properties

Finite number of sampies affect also symmetry properties of DFTs as

compared to those ofintegral Fourier transfonn:
 4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 181

U,v
{ aN -k
['2
exp - I ff
2v(k N-1/ 2)]} SDFT(u,v)
)

. 2u(r N-112)]} ;
{au,;~r exp[ 12ff (4.3.31)

u,v _ + u,v
{ a k - _a N -k
['2
exp - I ff
2v(k N-112 )J} SDFT(u,v)
)

{aU'v
r
=±au,v
N_
. 2u(r -1/
exp[ 12ff
N'
. 2)]} (4.3.32)

(4.3.33)

(4.3.34)

If 2u and 2v are integer,

Jj(_1)2V a } SDFT(u,v») Je 1)2u a }. (4.3.35)

~ N-2u-k l\ N-2v-r ,

Jflku,v = ±(_1)2v a N-2u-k

u,v }
pr = ±(-l\2u
SDFT(U,v») Lu,v
} aU'v }.
N-2v-k'
(4.3.36)

If 2v is integer,

If 2v is integer,
182 Chapter4

Eqs. 4.3.34 - 37 show that semi-integer shift parameters play an especial role
in symmetry properties of SDFTs.
For the DFT, above equations are reduced to

Ja
~ N-k } SDFT(u,v) ){aN-r }., (4.3.39)

J =+
l.ak _aN-k } SDFT(u,v) )JIV
l..... r = +IV }.
-U,N-r' (4.3.40)

(4.3.41)

{a k = ±(ak r} SDFT(U,v»

{ar = ±(aN-r)* ; } ;(4.3.42)

ao =(ao)*;ForevenN,aN/2 =(aN/2)*

{a k =(akr =±aN-k } SDFT(u,v) ){ar =±~N-rr =±aN-r }'

(4.3.43)

Symmetry properties of 2-D SDFTs may be obtained from the separability

ofthe transforms. For DFT, they are illustrated in Fig. 4-4.
Term ao plays a role of the zero frequency component of the integral
Fourier transform. It is proportional to the signal dc-component:

(4.3.44)
 4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 183

l-D signals:
EvenN OddN

k=O - ...~ 1 1f ........·IN/2 1 k=O I-...~ 1.·......··1

2-D signals:

Even N. k.=O
kz=O --+_. N.I2;O
f··· --+ f· ..
k.=O

-- ~
Even NI kz=O

~ I
~
Cl ~ -- -~

...··
O,N212 I N./2;
N z/2 I t ••• .. . -
~- .C2- ..J

--+ ....
k.=O
k:=t) ~

......I
k.=O
k1- O --+ N./2 öO ~.... ~ I
• .
~
......
-- - - -+ ....··
O.Nz/2 I
I
I

~- . - -
...... I
:
·
- - - - are complex congugate to

Figure 4-4. Types ofDFT spectra symmetries for real valued signals

For even numbered N, coefficient tZN 12 plays a role of the highest signal

frequency f =-N Ilf :

2

1 N-I
tZN/2= ~L(-lrak' (4.3.45)
vN k=O

For odd numbered N , coefficients tZ(N-I)/2 =(a(N+I)/2)· (for signals with

real va lues) plays a role ofthe highest signal frequency f =N -1 Ilf.

2
184 Chapter4

I-D signal, even N

Nil
I I• N-}

I-D signal, odd N

o 1 ~
••••••••••••
~ NIl-}

I I I 1(N-I)Il-1 I
•
(N+I)12 N-I o I J
••••••••••••

2-D signal, even N 2-D signal odd N

N./2; O,Nz/2 N,+112; 0,

N z/2
·:
x
N z++112 ··· N1+1I2

·· +.: ...... .... ...·· +· -

f 11: .... ~~,,* Il'J .. ~ ", ... ,.:a~ ... " :.;
·
~ ..... .....
N./2;O ::~:;i;o:: iN.+1I2; :::4~::
::(C~~~: ---t ---t
... !.... ..
0 ~
:~~:~

,.... ~ t- ,.... ~ +-
- - - are complex congugate to ••••• ••••• • ••••

Figure 4-5. Centering DFT spectrum (numbers in boxes indicate spectral indIces)

I 1' t I
i I1 I I I
Figure 4-6. Spectrum of a sinusoidal signal plotted without (central plot) and with (bottom
plot) spectrum centering
 4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 185

In order to maintain similarity with OFT spectrum {ar} of discrete signal

{a d and Fourier spectrum a{f) , f E [- 00,00] of the corresponding
continuous signal a{x), it is convenient to rearrange sequence {ar} as it is
shown in Fig. 4-5. Fig. 4-6 illustrates spectrum of a sinusoidal signal without
and with spectrum centering.

Discrete sampling theorem

Let discrete signal {ak} of LN sampIes be obtained from signal {ak N Jof
N sampIes (k N = O,l, •.. N -1) by placing between its sampIes (L-l) zero
sampIes:

(4.3.46)

where 60 is Kronecker delta (see Fig. 4-7 a" b). Let us compute OFT of
this signal:

(4.3.47)

where {ar },r=O,1,...,LN-1 is OFT ofsignal {a k }. Eq. (4.3.47) shows

that placing zeros between its sampIes results in a periodical replication of
its OFT spectrum (Fig. 4-7, c) with the number of replicas equal to the
number of zeros plus one (Fig. 4-7, d).
Let now multiply spectrum {ar} by a mask {,ur} that zeroes all periods
but one (Fig. 4-7, e). According to the convolution theorem, multiplying
signal OFT spectrum by a mask function results in signal cyc1ic convolution
with 10FT of the mask. In order to secure that the convolution kerne I will be
areal valued function, the mask should maintain spectrum symmetry for real
valued signals (Eq. 4.3.42). Therefore, for odd N, the mask should be:

Pr =l_rect r -(N +1)/2, (4.3.48)

LN -N-1
 186 Chapter4

a) ~)

Sincd-Interpolated signal
t)

Figure 4-7. Graphical illustration of the discrete sampling theorem: a) initial signal; b) OFT
spectrum of signal (a); c) initial signal with zeros placed between its sampies; d) its OFT
spectrum: periodical replication of the initial signal spectrum; e) removing all spectrum
replicas but one by the ideal discrete low pass filter; t) discrete sinc-interpolated signal
resulted from inverse OFT of spectrum (e)

where reet ( x) ={ I, 0< X< I} .

0, otherwise
4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 187

In this case, inverse DFT IDFT{[I- reet r -(N + 1)/2]tZ(r)modN}

LN-N-l
produces a signal

--
;;k = ~ LI-reet r-(N+l)/2]tZ(r)mOdNexP(-i21&-
1 LN-l[ kr) =
..;LN r=O LN -N -1 LN
1 [(N.;.!)/2 k )+
~ ~ Qr exp( - i21&-r
..;LN r=O LN
LN-l (, kr )]
). tZ(r )mod N exp - ,21&- =
r=LN~-1)/2 LN

~1 {(N.;!l/2[
~ 1 Lanexp
~ N-l nr)] ex{ -i21&-
( i21&- kr ) +
..; LN r=O ..; N n=O N LN
LN-l
r=r=Lf(N-l)/2
[1-La
.JN
N-l
n=O n
nr)] exp( -i21&-
exp( i21&-
N
kr )} =
LN
, (k -Ln,)
1{N-l
Ir L exp(-1211'
LQn [(N.:,!)/2 +
NvL n=O n=O LN

LN-l ' (k -
ex{-1211' Ln)-)]}
r=LN~-1)/2
-
LN-

ex [_i21&(k-Ln)(N +1)]_1
1 N-l P 2LN
-- La +
N JL n=O
n
exp[ -,'21& (k - Ln >] - 1
LN

ex{_;2.. LNt; Ln)] _ex{-;2K (k - LnlL~~ (N -1)/2)]

exp[ (k~:n)]_1 _i2..
=

ex.J- ;211' (k - Ln )(N + 1>] _ex.J_ ;211' (k - Ln )(N - 1>]

--LQn 1
1 N-l 2LN 1 2LN
=
NJL n=O exp 11' (k - Ln>] - 1
[ - I'2
LN
188 Chapter4

. [ N(k -Ln)]
1 sm • LN 1
I r La n SIncd[N,N,(k -nL)/L],
La n Nsin[2.(k _ Ln>] = vL
N-l N-l. •.
Ir
vL n=O n=O
LN
(4.3.49)

Using for k two indices k =Lk N + k L , we obtain finally:

(4.3.50)

This equation shows that, disregarding constant multiplier 11 JL , nonzero

sampies of signal 'iik remain, in signal i k , unchanged and equal to original
sampies ofsignal ~kN J
(4.3.51)

while its additional zero sampIes are interpolated with the discrete sinc-
function from the initial nonzero ones (Fig. 4-7, f).
For even N, term with index (N + 1)/2 required by Eq. (4.3.46) does not
exist and maintaining the spectrum symmetry is possible either with the
mask

P (O) =1- reet _r_-_N_I_2 (4.3.52)

r LN-N

that zeros N 1 2 -th spectrum component or with the mask

(2) 1 r - NI 2-1
Pr = - reet----- (4.3.53)
LN-N-2

that leaves two copies of this component in the extended spectrum. In these
two cases one can, similarly to the derivation of Eq. 4.3.49, obtain that
inverse DFT ofthe masked extended spectrum results in signals:
 4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 189

i(O)
k
=1DFTJ.·(O)a(
lI'r r) }= modN

r;- L ansined N -1;N;k -n+-k

1 N-l ( 1 )
N L (4.3.54)
~L~~ L

or, correspondingly,

(4.3.55)

Eqs. (4.3.50, 54 and 55) are fonnulations of the discrete sampling

theorem. It is a discrete analog of the sampling theorem for continuous
signals fonnulated in Sect. 3.3.1. Their dissimilarities are associated with the
finite number of signal and its spectrum sampies. Masking functions {,ur}
(Eqs. 4.3.48, 52 and 53) describe discrete ideallow pass filters. They are
different for odd and even N. The role of the sinc-function is played, for
discrete signals, by the discrete sincd-function defined in Eq. 4.3.18. If N is
an even number, original signal sampies are not retained in the interpolated
signal because of a special treatment required for the spectral coefficient
with index N / 2. In Sect. 9.2 we will show how the discrete sampling
theorem can be used for digital resampling signals specified by their
sampies.

4.3.3 Diserete eosine and sine transforms

There is a number of special cases of SDFTs worth of a special

discussion. All of them are associated with representation of signal and/or
spectra that exhibit certain symmetry. In digital image processing the
number of image sampies is most commonly even. If signals are
symmetrical with respect to a certain point, their corresponding discrete
signal with even number of sampies will retain signal symmetry if the
sampies are shifted with respect to the signal symmetry center by half of the
discretization interval. Therefore semi-integer shift parameters of SDFT
represent especial interest in this respect.
The most important special case is that of Discrete Cosine Transform
ta
(DCT). Let, for a signal k }, k =0,1, ..., N - 1 , fonn an auxiliary signal
190 Chapter4

_ = {a
ak
k, k = 0,1,•.., N - 1
(4.3.56)
a2N-k-l,k = N, ...,2N -1
and eompute its SDFT(l/2,0):

__ 1 2N - 1 _
SDFT1I2 •0 {a k } - ~
[.
L
a k exp 121&
(k+1I2)r]_-
",2N k=O 2N

~
1 {~l_ ['2
,,2N
~
k=O
a k exp , Ir (k+1I2}r] + 2~1_
2N k=N 2N
['2
~ a k exp , Ir
(k+1I2}r]} =

1 {N-l
~ Lak exp i21r
[ (k+1I2}r] + 2N-l
L
[(k+1I2)r]} =
a2N-k-l exp i21r
,,2N k=O 2N k=N 2N
~ {~la k exp[i21r (k + 1I2}r] + ~la k exp[i21r (2N - k -1I2)r]} =
,,2N k=O 2N k=O 2N
1 {N-l
~ Lak exp i21r
[ (k+1I2}r] + N-l
Lak exp -i21r
[ (k+1I2}r]} =
,,2N k=O 2N k=O 2N
2 ~l
~ ~ak
v2N k=O
[(k
cos I +r . 112']
N
(4.3.57)

In this way we arrive at the Diserete Cosine Transform defined as:

a rDeT __
- ~
2_~ [(k+1I2}r] .
~ak cos Ir (4.3.58)
,,2N k=O N

DCT signal speetrum is, as one ean see from Eq.(4.3.58), always an odd
(anti-symmetrie) sequenee ifregarded outside its base interval [O,N -1]:

a rDCT =-aDCT'a
2N-r' N
=0 (4.3.59)

while signal is, as it follows from Eq. 4.3.56, even (symmetrie):

(4.3.60)

From the above derivation of the DCT it also follows that, for the DCT,
signals are regarded as periodieal with period 2N :
 4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 191

(4.3.61)

The inverse DCT ean easily be found as the inverse SDFT(1I2,O) of anti-
symmetrie speetrum ofEq.(4.3.59):

- ,,2N1 2~1
Qk= LJa,DCT [.2
,=0
r=-:;
2N
exp-Iw (k+1I2'J =

La?!CT exp[(k+1I2),.J
-1- { afCT + N-l - i2w +
.J2N ,=1 2N
2~1 [.2
LJ a,DCT exp - 1 W(k+1I2'J =
,=N+l 2N
La?!CT exp[(k+1I2),.J
-1- { afCT + N-l - i2w +
.J2N ,=1 2N
~1
LJa,DCT exp[.2
-I W
(k + 1/2 )(2N - r >J =
,=1 2N
La?!CT exp[(k+1I2),.J
-1- {afCT + N-l -i2w +
.J2N ,=1 2N
~1
LJa,Dcr exp[ 1·2W(k + 1/2)- ] =
,=1 2N
-1- (afCT +2 N-l
La?!CT cos[ W (k+1I2)-JJ (4.3.62)
.J2N ,=1 N

Coeffieient afCT is, similarly to the case of DFT, proportional to the signal
de-eomponent:

afCT =.J2N ( -1 N-l

LQk . J (4.3.63)
N k=O

Coeffieient a!~i

a NDer
_1
_
-
2 ~ (Ir.sm(1f k+ 112)
~Qk -
r;;-;;; (4.3.64)
...;2N k=O N

represents signal's highest frequeney (compare with Eq. 4.3.45 for DFT).
192 Chapter4

The DCT ean be regarded as a diserete analog of integral eosine transform

(Eq.2.4.24). It was introduced by Ahmed and Rao ([1]) as a transform for
image coding. It has a very good energy compaction property and is very
frequently considered to be an approximation to Karhunen-Loeve transform
([2]). From the above derivation, it becomes eIear that the energy
compaction property of the DCT has a very simple and fundamental
explanation. It should be attributed to the fact that the DCT is a shifted DFT
of a signal that is extended outside the base interval [0, N -1] in a way
(4.3.56) that removes the signal discontinuity at the signal border, the main
cause of poor convergence property of DFT and other bases defined on a
finite interval. The absence of discontinuity at signal borders makes the DCT
to be attractive not only as a good transform for signal discretization but also
as a substitute for DFT for the implementation of signal digital convolution
in transform domain with the use of Fast Transforms. We will discuss this
application in Sect. 5.2.
The DCT has its complement sine trans form, the DeST:

"0 DcST 2 ~ . [ (k +1I2}-J .

= ,,2N
r:;:; ~ak sm fl (4.3.65)
M N

The DcST is an imaginary part ofthe SDFT(1I2,O) ofa signal

_ ={a
ak
k , k = 0,1,••. , N -1
(4.3.66)
-a2N-k-l' k=N,N +1,...,2N-l

extended to interval [0,2N -1] in an odd symmetric way:

1 2~1_ak exp['2
r:;:; ~ , fl (k+1I2)rJ =
i,,2N k=O 2N

1 {~l_ ['2 (k+1I2)rJ + 2~1_

~ak exp , fl
r:;:; ['2 (k+1I2)rJ} =
~ ak exp , fl
i" 2N k=O 2N k=N 2N

1
. r:;:;
{N-l
Lak exp[{k+1I2)rJ
i2fl -
2N-l
L a2N-k-l exp[{k+l/2)rJ}
i2fl =
'" 2N k=O 2N k=N 2N

1 {~1
r:;:; ~ak exp['2
I If (k+1I2}rJ -
~1
~ak exp['2
, If (2N-k-1I2)rJ} =
i,,2N k=O 2N k=O 2N

1
. r:;:;
{N-l
Lak exp[{k
i21f +1I2)rJ - N-l
Lak exp[ - i2fl {k +1I2)rJ} =
'" 2N k=O 2N k=O 2N
 4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 193

2 La
r;;;-;:;
N-I
k sm
• [
tf
(k+1I2,] . (4.3.67)
,,2N k=O N

DeST s,peetrum exhibits even symmetry when regarded in interval

[O,2N -1J:

a rDcST -_ a 2N-r·
DcST (4.3.68)

and assumes periodieal replieation, with aperiod 2N, of the signal

eomplemented with its odd symmetrieal eopy:

(4.3.69)

In distinetion from the DFT and the DCT, the DeST does not eontain
signal' s de-eomponent:

(4.3.70)

and the inverse DeST :

a
k
= -t- ( tzDcST + 2 N-l
.J2N
~ tz DcST sin[ tf
N ;:. r
(k+1I2'JJ
N
(4.3.71)

involves speetral eoeffieients with indices {t,2,..., N} rather then

{O,I,..., N -I} for the DCT.
In a similar way one ean introduee different modifieations of the SDFT
for different shifts and different types of signal symmetry. For instanee,
SDFT(1I2,1I2) of a signal extended in an anti-symmetrie way (Eq. 4.3.66) is
a modifieation of the DCT

(Zr
DCTIV _
-
&~l
- LJak cos [(k + 112)(r + 112>]
tf-L-_--.l.C1.-_--L (4.3.72)
N k=O N

that obtained a name DCT-IV ([3]). One ean easily verify that the DCT-IV
speetrum also exhibits an odd symmetry:

LDCTlV _ __ DCTIV }
pr - -2N-r-1 . (4.3.73)
 194 Chapter4

Therefore the inverse DCT-IV is identieal to the direet one

DCTIV _
ak -
&~1 [(k + 1I2Xr + 11 2)] .
- ~a, cos Il (4.3.74)
N ,=0 N

Beeause of 1I2-shift in frequeney domain, the DCT-IV of a signal assumes

periodieal replieation, with altemating sign, of the auxiliary signal of Eq.
4.3.66.
Similarly to the DCT-IV, one ean define the DST-IV ([3]):

a,DSTIV =&~1
-
N
~
k=O
.[
a k sm (k+1I2 Xr+1I2)]
Il~-----"=-_--L
N
(4.3.75)

that is obviously an imaginary part of the SDFT(I/2,1/2) of a symmetrized

auxiliary signal ofEq. 4.3.56.
In the same way one ean show that the SDFT(1, 1) of an auxiliary signal
{ak} formed from a signal {a k, k =0,1,..., N - I} by extending it in an odd
way to 2N + 2 sampIes:

a k , O~k~N-l
-
ak =
0,
-a 2N - k ,
k=N
N~k~2N
(4.3.76)

0, k=2N+l

reduees to the transform

DST _
a,
1
-~ ~aksm Il
~1 • [ (k+1 Xr+l)] (4.3.77)
,,2N k=O N

known as Discrete Sine Transform (DST). DST speetrum is also and in the
same way anti-symmetrie:

J_DST _
Pr -
-a2N-r'
DST }. a DST _ a DST - 0
N - 2N+l- (4.3.78)

and, therefore, inverse DST is also identical to the direet one:

 4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 195

ak -
_ ~
1 ~l
~ar
DST • [
sm Ir
(k + 1)(r + 1)] . (4.3.79)
,,2N r=O N

DST of a signal assumes periodical replication, with aperiod 2N + 2 of the

auxiliary signal ofEq. 4.3.76.
2-D and multi-dimensional DCTs and DSTs are usually defined as
separable to 1-D transforms on each of the coordinates. For instance, 2-D
DCT of a signal ta k,1 J ' k =0,1, ..., NI -1, I =0,1, ..., N 2 -1 is defined as

that corresponds to 4-fold image symmetry illustrated in Fig. 2-15.

4.3.4 Lapped Transforms

Discrete orthogonal transforms such as DFT, DCT and DST can be applied
to signals either globally, i.e. to all available signal sampIes, or block wise or
in a sliding window. If they are applied block wise, signal is subdivided into
blocks and the corresponding transform is applied to individual blocks. In
block transform signal processing, independent treatment of individual
blocks may cause so called blocking effects or artifacts on the borders of
blocks. Radical solution of this problem is processing in transform domain in
a sliding window. We will discuss application of sliding window transform
domain processing in Chs. 8 and 9. Another option that allows to
substantially reduce blocking effects is signal processing in hopping
overlapping windows. Most popular and computationally efficient is half
window overlapping. Transforms that are used in this processing are called
"Lapped" transforms ([4]).
One of the most popular lapped transforms is Modijied DCT (MDCT) or
Modulated Lapped Transform (MLn. The MDCT of a signal sequence
{a k }, k = 0,1,.••,2N -1 is defined as

a~DCT = fN
{22N-l N -
~ akhkcos[II( k+-2
+1)( 21)/]
r+ N (4.3.81)

where {hk J is a window function:

. ( Ir k
h k =sm +1/2J
N
. (4.3.82)
196 Chapter4

One can show that MLT is equivalent to the SDFT ( N + 1 ,!. ) of a signal
2 2
{ak = akhk} modified in a certain way:

(4.3.83)

Replacement of summation index k in the second and fourth tenns of Eq.

4.3.83 by(N -1- k) and (3N
-1- k) , respectively, yields:

a:=' =~2~ ~a.ex+++N;lXr+i)/N]+

&~aN_I_.e~-i'[2N ++ N;1)](, +i)/N} +
J2~ :~lak exp[i.(k N; l)(r i)/N]
++N_( +N; )]('+
+ + +

&:~I a'N_I_. ex i k 1 i)/N}~

&~.kex+++N2+1X'+i)/NJ+
 4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 197

~~aN-I-k exp[ -i21r(r +~J}XP[i1r(k +N2+ 1J(r +lJ/N] +

~:~I .. ex+++ \+l)(r +i)/N]+
&:~la3N_I_k ex{-i4~r+~J}XP[i1r(k +N2
+1J(r+1J/N].
(4.3.84)

With substitution ex p[ - i21r( r+ ~)] = -1, exp[ - i41r( r+ ~)] =1 and

_ {ak-a N-1-k' k=O,I, •.. ,N-l
(4.3.85)
ak = ak + a3N-l-k' k = N,N + 1,•.•,2N-l

obtain finaIly:

a,MDCT = .J2N 1 2~1_

f:o akhk exp[ .I1r ( k +- +1)( +"21))]
N2 - r N ,(4.3.86)

the SOFT ( N + 1 ,! ) of a signal {äk} fonned from the initial windowed

2 2
signal {ak} according to Eq. 4.3.85. Physical interpretation of Eq. 4.3.85 is
straightforward. According to it, first half of the signal sampies is mirrored
and overlaid onto the signal in negative and second half of the signal
sampies is mirrored as weIl but overlaid onto the signal in positive. In signal
processing with MLT, successive window positions overlap by half of the
window of 2N sampies. With this overlapping, same mirrored halves of the
signal within window overlaid, in two successive window positions, with
opposite signs and compensate each other. This seeures perfeet
reconstruction of the signal.
It follows from Eqs 4.3.81 and 4.3.86 that spectral coefficients of the
MOCT exhibit two-fold redundancy:

a, =(-1 )N+l a2N-l-r. (4.3.87)

198 Chapter4

From Eq. 4.3.86 it follows that the inverse MDCT coincides with the direct
one:

äk 1
= .J2N N-I N -+
f:otz~DCT cos[Ir( k+- 2
1)( 1)/]
r+2" N (4.3.88)

and that the MDCT of a signal {a k} assurnes sign alternating periodical

replication with period 2N ofsignal äk ofEq. 4.3.85:

(4.3.89)
4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 199

4.4 DISCRETE REPRESENTATION OF FRESNEL

INTEGRAL TRANSFORM

4.4.1 Discrete Fresnel Transform

Transformation kerne 1 of integral Fresnel transform exp~ in(x - f)2 is J

shift invariant. Nevertheless, in the transform discrete representation, it is
useful to account for possible arbitrary shifts of signal and its Fresnel
spectrum sampies positions with respect to the signal and spectrum
coordinate systems in a similar way as it was done for discrete representation
ofFourier Transform in Sect. 4.3.1.
Let signal reconstructing and spectrum discretization basis functions be
.,Y>(x)= I> [x -(k + u}1x], and .,~d>(x)= Jd>[r -er + v}V] , where k
and r are integer indices of signal and its spectrum sampies and u and v
are, respectively, shifts, in fractions of the discretization intervals Ax and
fit , of signal and its spectrum sampies with respect to the corresponding co-
ordinate system. Then discrete representation of the Fresnel transform kernel
is, according to Eq. 4.1.6,

-00 -00

-00 -00

(4.4.1)

Introduce, temporarily, a variable

(4.4.2)

Then

-00 -00

-00
200 Chapter4

jtp(r)(x)exp(- itrx 2)exp[i2trx(f - Srk)}Ix =

-00

-co

where

(4.4.4)
-00

is Fourier Transform of signal reconstruction basis function tp(r )(x)

modulated by the chirp-function. For tp(r )(x) is a function compactly
concentrated around point x = 0 in an interval of about ßx ,
exp(- itrx 2)== 1 within this interval and one can regard <i>(f)as an
approximation to frequency response of a hypothetical signal reconstruction
device assumed in the signal discrete representation. With similar
reservations as those made for discrete representation of Fourier integral, for
discrete representation of Fresnel integral only term

(4.4.5)

ofEq. 4.4.3 is used.

To complete the derivation, introduce dimensionless variables:

(4.4.6)

and

w=up-v/p. (4.4.7)

Then, using relationship N = 1/ßxllt between the number of signal

sampies and discretization intervals (Eq. 4.3.2), obtain for hk,r finally:

hk,r =exp~ ilr(k + u}1x - (r + v~f]2 }=

4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 201

exp{- itr [(k + U }1x - (r + v)v]2 } = exp[- itr ..3....!(kp'---_r---=.1p_+_w-L-)2]

N~Af N
(4.4.8)

With this discrete PSF, basic equation Eq. 4.1.5 of digital filtering takes, for
the representation of Fresnel Transform, the form:

I •• )
a~'w = - - ~ a
1 N-l
exp[ -itr (kP-rlu+w)2]
r (4.4.9)
r '-;;;N
~ k N
"1'1 k=O

where multiplier 1 I JN is introduced, similarly to Oiscrete Fourier

Transforms, for normalization purposes. Eq. 4.4.9 can be regarded as a
discrete representation of the Fresnel integral transform ([5]). We will refer
to it as to Shifted Discrete Fresnel Transform (SDFr1). Its parameter p
plays a role of the distance parameter AD in the Fresnel approximation of
Kirchhof's integral (Eq. 2.3.10). Parameter w is a combined shift parameter
of discretization raster shifts in signal and Fresnel Transform domains. It is
frequently considered equal to zero (see for instance, [6,7]) and Discrete
Fresnel Transform (DFr1) is commonly defined as

1 N-l [ Ckp
aY',w)=--Lakexp-itr
JN k=O N
\l]
- r I p}. (4.4.10)

SOFrT (p, w ) is quite obviously connected with SOFTe O,-wp ):

a~'w) 1 {N_l
=-- La k 1p1) [
exp(-itr--
p
exp i2tr k(r - w )]} x
JN k=O
k
N N

exp[- itr (r - W~)l] (4.4.11)

Np

From this relationship one can conclude that SOFrT is invertible and that
inverse SOFrT is defined as

I..)
a~'w
1 N-l
=__ I..) [
~a~'w exp itr
(kp - r lu
r.
+W )2] (4.4.12)
k JNf::, r N
202 Chapter4

Because shift parameter w is a combination of shifts in signal and

spectral domains, shift in signal domain causes a corresponding shift in
transform domain which, however depends on the focal parameter JI
according to Eq. 4.4.7. One can break this interdependence if, in the
definition of the discrete representation of integral Fresnel transform, impose
a symmetry condition:

(4.4.13)

of the transform of a point source 8(k)

(4.4.14)

This symmetry condition is satisfied when

N
W=- (4.4.15)
2p

and SDFrT for such shift parameter takes form:

(",;,.) _
(Zr -
1 ~
r;:; 4.- ak exp -u&
vN k=O
{ . [kp-(r-N /2)/
N
pr} . (4.4.16)

Transform defined by Eq. (4.4.16) allows to keep position of objects

reconstructed from Fresnel holograms invariant to focus distance. We will
cal this version of the discrete Fresnel transform Focal Plane Invariant
Discrete Fresnel Transform .
Discrete representations of the partial Fresnel Transform are obtained by
removing, from Eqs. 4.4.9 and 16, exponential phase terms that do not
depend on k. Therefore, Partial Discrete Shifted Fresnel Transform is
defined as

a~'w) = --La
1 N-I k 1p1 ) exp[ i21& k(r-wp )] .
exp( -i1&-- (4.4.17)
JNk=O k
N N

with its inverse transform as:

 4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 203

ar'w) = _1_{~a~'W)eXP[_i21&(r - wp}kJ}eXP(il& k

1p1 ). (4.4.18)
JN r:O N N

and Focal Plane Invariant Partial Discrete Fresnel Transform as

(4.4.19)

with its inverse transfonn as

a" = 1
r,;; (_l)k exp(k1p1)N-I_ (
- il&-- La~ exp -
krJ
ill&- . (4.4.20)
"\IN N k:O N

Fig. 4-8 illustrates using Discrete Fresnel and Focal Plane invariant
Discrete Fresnel Transfonns for reconstruction of optical holograms.

Figure 4-8. Reconstruction of a hologram recorded in a form of amplitude and phase. a) - c):
reconstruction on different depth using Discrete Fresnel Transform (Eq. 4.4.10); d) - f) -
reconstruction using Focal Plane Invariant Discrete Fresnel Transform (Eq. 4.4.16). (The
hologram was provided by Dr. G. Pedrini 3,[8])
 204 Chapter4

Fig. 4-9 illustrates optical reconstruction of a computer generated

hologram that combines holograms computed with the use of Discrete
Fourier and Discrete Fresnel Transforms. As one can see, the hologram is
capable of reconstructing different object in different planes. One can also
see in the images zero order diffraction spot that appears in reconstruction of
holograms recorded on amplitude-only medium (See Sect. 13.3).

Figure 4-9. Optical reconstruction of a computer generated hologram computed using

Discrete Fourier Transform and Discrete Fresnel Transform for visualizing objects in different
planes: a), c) - front and rear focal planes ofthe Fresnel transforrn; b) - Fourier plane (adopted
from [6]).
 4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 205

4.4.2 Properties of Discrete Fresnel Transforms

lnvertibility and frincd-function

For a discrete signal {a k },

k = 0,1,••• , N -1, let us compute inverse
SDFrT with shift parameters (p_, w _) of its SDFrT spectrum computed
with parameters (P+, w+):

( .. ±
ar'
w±)
=
1 ~1
r.; ~a'f'
, __+ w+) {. [kp_ - r I p_ + W
exp Itl
-1 2}
=
..,;N r=O N

-1 ~l{ - 1 ~l
~ ~a
[.
exp -Itl (np+ - r I P+ + W +)2 ]} x
JN r=O JN n=O n N

. (kp_ - r I p_ + w _ )2 ]
exp[ Itl =
N

~ ~l~lan eXP{itl[(np + -rl P+ + w+Y -(kp_ -rl p_ + w_Y]}=

N r=On=O N

~ exp[- itl (kiL + w - )2]~1 a n exp[itr (np+ + w + )2] X

N N n=O N

N-l [ r2 2 _ 1/u 2 )j
(1/ur~ ( n-k+w+;+
lu -w-lur- ) .
~ exp itr r- exp - i2nr

(4.4.21)

Introduce the following substitutions:

1/P: -1/p~ ;
N _ N _
w+ =--+w+; w - =--+w
2 -,. q=
2p+ p-
w± = w+lp+ - w_Ip_ . (4.4.22)

and obtain finally:

1
a~± ,w±) = -exp
. k +W
-Itl p-
N
- [ ( N
)2] Ia
N-l

n=O
n
[(nP+ + w )2] x
exp Itr
•

N
+
206 Chapter 4

l: eXp[if& rq(rN- N >] exp(- illfl' n - k + w /11+

r~O
+
N
- w-/11- ) =

~ex{_i.. (kP- ;W-Y]X

~ .. ex+.. (np, ; wJ ] frinCd(N; q; n- k + w,), (4.4.23)

where

frined(N; q; x)= ~
N~
l: exp[if& qr(rN- N )] exp(- ilf& xr)
N
(4.4.24)

is a frincd-function, a discrete analog of the frinc-function of integral

Fresnel Transform (Eq. 2.4.51). It is also an analog of sincd-function of the
DFT and is identical to it when q =0 :

frined(N ;0, x)= sined(N; N; x) (4.4.25)

In particular, when q =0, from Eq. 4.4.23 obtain:

.~'·)ex{_i.. (kP-; wS]x

~ •• ex+.. (np , ; w,Y }InCd(N;N;n-k + w,) (4.4.26)

Thus, the inconsistency only in direct and inverse transforms shift

parameters results in reconstruction of a chirp-demodulated sincd-
\w
interpolated, w± = + - W - }/11- shifted initial signal modulated by a chirp-
function.
Parameter q of frincd-function plays a role of a focussing parameter
associated with distance parameter XD in near zone diffraction equation (Eq.
2.3.11) modeled by the integral Fresnel Transform. From Eq. 4.4.24 one can
see that frincd-function is a periodical function with period of N sampIes:

frined(N;q,x)= frined(N;q,(X)modN) (4.4.27)

and that
 4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 207

(frincd(N; q,x»" = frincd(N;-q,N - x); (4.4.28)

1 N-l qr(r - N»)

frincd(N;q,O)=- ~:exp ilr
(
. (4.4.29)
N r=O N

Fig. 4-10 illustrates the behavior of frincd-function for different values of

focussing parameter q.

abs(frincd)
q=O •

0.9 _ . . • • . , . • . . •. ••• . . • . .• • . . •• • . • . . •. .

0.8 ............. , ...... " ..... , ..... .

0 .7
. . .
. • • • . . • • . . . . . ; . . . • . • ,. . • • • . ; • . . • . •
.
0.6 ................ .. ......... ~ ........................... : .... .. ..... ..

0.5 ........... ' .... .. .... . . , . .. .

.
q:,Q.02 • • • • •
. , • • • _.
• •
.
0 .4 ... . .••......•..•

0.3 _ . • • • . , . • • . •• ; • • . ... . . - . ....

.... .. . .... ...... ' .

0.2

0 .1

Figure 4-10. Absolute values of function jrincd (N;q;x) for N=256 and different va lues of
"focussing" parameter q. (The function is shown shifted to be centered around the middle
point ofthe range ofits argument)

Periodicity:
F or integer hand g:

ak+ Nh
. 2(k +"2
=a k exp[12f1hp w)] .
Nh + p (4.4.31)

(4.4.32)
208 Chapter 4

Signal and Fresnel spectrum sinusoidal modulation and shift:

a k-k o exp[- i1&

(k_k)2
0
2]
I' ( )ar exp[i21& ko{r: Wl')].
lV

(4.4.33)

. {r1'2
a r exp[11& - ro
lV
r] (
SDFrT
".w) a k exp[ -121&
• ro(k +lVw / 1')] .(4.4.34)

SDFrTs ofselected elementary signals:

L 6 } SDFrT".w
l"k = k-k o (

--(

JN
1 ISDFrT",w
frinCd(lV; ~; k+ w _ ~J exp[i1& (k + lV/21'2
1'2 I' 21'2 lV
r]
(4.4.37)
4. DIGITAL REPRESENTATION OF SIGNAL TRANSFORMATIONS 209

References

1. N. Ahmed, K.R. Rao, Orthogonal Transforms for Digital Signal Proeessing, Springer
Verlag, Berlin, Heidelberg, New York, 1975
2. A. K. Jain, Fundamentals of Digital Image Proeessing, Prentiee Hall, Englewood Cliffs,
1989
3. Wang, Z., Fast Algorithms for the diserete W transform and for the diserete Fourier
Transform", IEEE Trans. Aeoust., Speech, Signal Proeessing, vol. ASSP-32, Aug. 1984,
pp. 803-816
4. H. S. Malvar, Signal Proeessing with Lapped Transforms, Artech House Ine., Norwood,
MA,1992
5. L. Yaroslavsky, M. Eden, Fundamentals ofDigital Opties, Birkhauser, Boston, 1996
6. L. Yaroslavsky, N. Merzlyakov, Methods of Digital Holography, Consultant Bureau,
N.Y., 1980
7. T. Kreis, Holographie Interferometry, Prineiples and Methods, Akademie Verlag, Berlin,
1996
8. G. Pedrini, H. J. Tiziani, "Short-Coherenee Digital Mieroseopy by Use of a Lensless
Holographie Imaging System", Applied Opties-OT, 41, (22),4489-4496, (2002).
 Chapter 5

METHODS OF DIGITAL FILTERING

5.1 FIL TE RING IN SIGNAL DOMAIN

5.1.1 Transversal and recursive filtering

As it was stated in Sect. 4.2, basic fonnula of digital filtering a signal

defined by its sampIes {a k , k = 0,1,•.. , Na -I} by a filter defined by its
discrete PSF {h n , n =0,1, ..., N h - I} is

Nh-I

bk = Lhna k - n (5.1.1)
n=O

Direct computational implementation ofthis fonnula is calledjiltering in

signal domain.
An important issue in digital filtering is its computational complexity. In
conventional serial computers filtering in signal domain is carried out by
signal weighted summation according to Eq. 5.1.1 in sliding window of N h
signal sampIes as it is illustrated in Fig. 5-1. This filtering technique is
referred to as transversal jiltering. The computational complexity of
transversal filtering according to Eq. 5.1.1 is defined by the extent of digital
filter PSF: N h addition and N h -1 multiplication operations per output
signal sampie. This complexity may be reduced if to use, in course of
filtering in each filter window position, filtering results obtained on previous
positions. Such filtering technique referred to as recursive jiltering is
described by equation:
212 Chapter5

!I
\ I
\0 \: .I I Direction of signal .anning
~ ~ ••••••••• I

11+.1
Ili,r.,.II,I"I. Output signal sampies

Figure 5-1. Signal filtering in sliding window

(5.1.2)

where N!t) and N~r) are the numbers of addends in the transversal (first
sum) and recursive (second sum) parts, respectively, of the filter. In
hardware implementation, recursive filters correspond to feedback circuitry.
Eq. 5.1.2 assumes that, for computing each output signal sampie, N!t)
input signal sampies andN~r) previously computed output signal sampies are
used. To illustrate computational saving with recursive filtering consider the
simplest case, when N!t) = 1 and N~r) = 1 :

(5.1.3)

Computations according Eq. 5.1.3 require two multiplications and one

addition operations, while the same filter in the transversal form:
 5. METHODS OF DIGITAL FILTERING 213

may contain unlimited number of terms.

A good practical example of the recursive filter is the filter that computes
signallocal mean in a window of 2N + 1 sampIes:

1 N
b - ~a (5.1.5)
k - 2N + 1 L.J k-n'
-N

Eq. 5.1.5 can be represented in a recursive form

(5.1.6)

that requires two addition and one multiplication operation independently on

window size while, in its transversal form, filter requires 2N + 1 additions
and one multiplication.
Computational saving possible with recursive filtering suggests that
whenever possible, the desired filter should be approximated by a recursive
one.
In fiItering 2-D signals, there is no natural sequential ordering of signal
sampIes. The definition of "past" and "future" sampIes in sequential signal
processing is a matter of a convention. Usually, 2-D discrete signals are
treated as 2-D arrays of numbers and sequential filtering is carried out row
wise from top to bottom rows of the array. Arrangement of "past" (bygone)
and "future" sampies in such a signal scanning is illustrated in Fig. 5-2 .

...... .
~
•••••••••••
• • • • • • • • • • •
. 0000
"Past" sampIes

"Curren t" samp Ie ~ 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0
"Future" sampies
00000000000

Figure 5-2. "Past" and "future" sampies on a rectangular sampling raster at 2-D row wise
filtering

5.1.2 Separable, cascade, parallel and parallel recursive filtering

General digital filtering of a 2-D signal defined by its sampIes {ak,/} is

described by the equation:

N-IM-I

bk,1 = L L hn,mak-n,l-m . (5.1. 7)

n=Om=O
214 Chapter5

Oirect computations according to this formula require NM multiplication

and summation operations per ou~ut sampie. This number can be
substantially reduced if filter PSF {h",m} is a separable function of its
indices:

(5.1.8)

In this case filtering can be carried out separately row-wise and column-wise
with corresponding 1-0 filters

(5.1.9)

which requires N + M multiplication and summation operations rather then

MN ops. In Chapt. 6 we will see that representation of transform in a
separable form is a key point for synthesis of fast transform algorithms.
Separable representation of digital filters is an example of decomposition
of complex filters into a set of more simple and fast ones. Two other
methods for such a decomposition are cascade and parallel implementation
of filters. In cascade implementation, desired filtering is achieved by
sequential application of a number of sub-filters as it is illustrated in the flow
diagram in Fig. 5-3.

--+I Filter 1 H H. _F_il_te_r_3---'~""'~L._F_il_te_r_R---,~

Filter2

Input signal Output signal

Figure 5-3. Tbe principle of cascade filtering

PSF of a filter implemented as a cascade of R sub-filters with PSFs

~!,), r =t,2, ...,R} is a discrete convolution ofPSFs ofthose filters:

(5.1.10)

As it follows from the convolution theorem of the OFT, discrete frequency

response ofthe cascade of R sub-filters is a product
5. METHODS OF DIGITAL FILTERING 215

DFR(P)= rr DFR,(p).
R

,=1
(5.1.11)

ofdiscrete frequency responses {DFR,(P)} ofthe sub-filters defined by Eq.

4.2.10.
Factorizing DFR of complex filters into a product of DFRs of more
simple sub-filters is a way to design cascade implementation of complex
filters. A good example of the cascade implementation of filters are splines.
In spline theory terminology ([1,2]), signal moving average filter (Eq. 5.1.5)
is a zero-order spline. First order spline is obtained as the convolution of two
zero-order splines. Its PSF is obviously a triangle. Second order spline is the
convolution of the first and zero-order splines. Higher order splines are
corresponding higher order convolutions.
Let us evaluate potential computationa1 gain that can be achieved by
cascade filtering with respect to direct filtering. Consider first 1-D filters. Let
N is the number of sampies in PSFs of sub-filters. Each sub-filter in the
cascade after the first one adds to this number N -1 sampies. Then, for
R sub-filters in cascade, equivalent PSF will have R(N -1)+ 1 sampies.
This figure defines the computationa1 complexity of the direct filtering while
the computationa1 complexity of the cascade filtering is defined by the sum
NR of the number of sampIes in PSFs of sub-filters which is higher then
R(N -1)+ 1. Therefore, the computational complexity of the cascade
implementation of 1-0 filters may even be higher then that of direct filtering
unless sub-filters are recursive ones. For 2-D filtering, the situation is
completely different. If NM is the number of sampies in PSFs of R 2-D
sub-filters in cascade, the equivalent number of sampies in PSF of the entire
cascade that defines its com.rutational complexity in the direct filtering is
[R{N -1)+ l]x [R(M -1)+ 11- The complexity of the cascade filtering is
defined by the sum RNM of PSF sampies of R sub-filters. Therefore the
computational complexity of2-D cascade filtering is approximately R times
less then that of the direct filtering.
Parallel implementation of filters assumes filter decomposition into a
number of sub-filters that act in parallel (see Fig. 5-4).

R-inputs adder
1-.-_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _...... output signal

Figure 5-4. The principle ofparallel implementation offiltering

216 Chapter5

For such an implementation, filter PSF {h,.} has to be decomposed into a

sum

R
h,. =~ h(')
~,. (5.1.12)
,=1

ofPSFs ~!,), r = 1,2,...,R}

ofsub-filters.
Parallel filtering is weIl suited for parallel computing. In serial computers,
it can be carrled out by sequential signal filtering with sub-filters and
subsequent addition of the filtering results. Computational saving can in this
case be achieved in this case only if sub-filters are recursive.
Let us show how an arbitrary digital filter can be represented in such a
parallel and recursive form. Consider a digital filter:

(5.1.13)

and assume that filter PSF {h,.} can be approximated, with the desired
accuracy, by expanding it into aseries over a system of basis functions
~,(n} r=1,2, ...,R; RSNfh)+N~h)+l}:

R
h,. == L'1,f',{n}. (5.1.14)
,=1

Then filter output signal can then be approximated as

N~') R R Nlh ) R

b k == _~A)ak-"~l'1,,,,(n)= ~1'" _~t,(n}zk-" =~",p,(k),

(5.U5)

where

(5.1.16)
 5. METHODS OF DIGITAL FILTERING 217

Eq. 5.1.15 describes signal filtering with sub-filters with PSFs ~r(n)} in R
parallel branches and adding up the sub-filtering results with weights Wr}.
Let us now find conditions under which coefficients fJ r (k) in branch
filters can be computed recursively. In the simplest first order recursive
filter, every given output signal sampie is computed from the output sampie
determined at the preceding filter window position. Let us establish a link
between {P,(k)} and {P,(k -I)}:

and choose "r(m) such that

(5.1.18)

Then obtain:

Eq. 5.1.19 signifies that when basis functions {tp,(n)} satisfy Eq. 5.1.18
coefficients fJ r (k) can be computed recursively with the recursiveness on
one preceding step.
It follows from Eq. (5.1.18) that the class of the recursive basis functions
is that of exponential functions:

(5.1.20)

Two most important special cases are the basis of discrete exponential
functions:
218 Chapter5

(5.1.21)

where Po is a free parameter and the linearly independent basis of

rectangular functions

(5.1.22)

which can be obtained from Eq. (5.1.21) when Po =0. The latter
corresponds to recursive computation of signallocal mean:

When the basis of Eq. 5.1.21 with P =1 is used each of the R parallel
recursive filters (5.1.16) performs signal discrete Fourier analysis in the
window of M = Nfh) + N!h) + 1 sampies at the frequency r. The resultant
spectral coefficients are multiplied element-wise by the spectral coefficients
of the filter pulse response, i.e. by sampies of its discrete frequency
response, and summed up in order to obtain the filtering result. The basis
of Eq. 5.1.21 with Po =112 corresponds to Iocal DCT/DcST analysis.
Corresponding recursive algorithms of sliding window DFT and DCT/DcST
are discussed in Sect. 5.1.3
When the basis (5.1.22) is used, the digital filter is composed of a set of
loeal mean filters with different window size. The impulse response of the
filter is a sum of R rectangular functions of various extents. Synthesis of
such a representation for a particular filter is therefore reduced to
quantization of values of its PSF to R quantization levels (Fig. 5-5). For
minimization of filter PSF the approximation error caused its quantization,
optimal quantization methods discussed in Sect. 3.5 may be used.

Figure 5-5. An example ofa filter PSF (bold bars) and its quantized approximation with 8
quantization levels (c1ear bars)
 5. METHODS OF DIGITAL FILTERING 219

For two-dimensional digital filters,

(5.1.24)

basis functions f""s(m,n)} in the expansion of the filter PSF

R-IS-I
hm,n ~ L L'I"sf'"s(m,n) (5.1.25)
,=os=o

are represented in the simplest possible way as separable functions

f'"s(m,n)= f'~I)(m )p~2)(n). The associated parallel filters

(5.1.26)

would then be the simplest possible. The above results are obviously
applicable to the one-dimensional functions f'~I)(m) and f'~2)(n). Hence in
the two-dimensional case, "pure" bases of exponential and rectangular
functions may be supplemented by "mixed" bases, rectangular in one
coordinate and exponential in the other coordinate.
There are also inseparable recursive two-dimensional bases. One of
them is a basis of rectangular functions within an arbitrary shaped area on a
rectangular raster. An analogous separable basis is bounded by a rectangle.
The aperture support of an inseparable two-dimensional filter is shown in
Fig.5-6, b) and its separable analog is shown in Fig.5-6, a) in which the
boundary elements of the aperture are cross hatched.

a) b)

Figure 5-6. Pattern of support of separable (a) and inseparable (b) 2-D moving average bases
(pixels that are drop out the filter aperture are single line hatched, those that enter it are
double line hatched).
220 Chapter 5

5.1.3 Recursive algorithms ofsliding window DFT and DCTlDcST

Let us consider OFT of two adjacent (k -1 )-th and k -th fragments

~k-I-f1X«Nw-I)/2),.··ak-l+f1X(NwI2J
and {ak-flX«Nw-t)12),. .. ak+flX(Nw/2)} in sliding
window of N", sampIes of a signal {ak}

ar
(k-l) _
-
1
r;r
'\IN",
L
N w -I

n=O
(
• nF
)
ak-t-flX«Nw-t)/2)+n exp ,21&-
N",
(5.1.27)

and

(5.1.28)

where fa{x) is integer part of x. Spectrum ~~k)} in k -th position ofthe

sliding window can be computed from spectrum ~~k-l)} in (k -1)-th
window position in the following way:

Eg. 5.1.29 describes the recursive algorithm for computing DFT signal
spectrum in sliding window of N", sampIes that reguires, for signals whose
sampies are real numbers, 2(N", -1) multiplication operations and (N", + 1)
addition operations.
 5. METHODS OF DIGITAL FILTERING 221

The DCT and DcST in a running window can be computed in a similar

way ([3]). For adjacent (k -I)-th and k -th signal fragments, their DCT and
DcST spectra ~~k-t)}and ~~k)}are, correspondingly, real and imaginary
parts of

a,
(k-t) __I_~
- ~ -'..Jak-t-ftx«N",-t}i2)+-n exp ur
(0
(n+1I2)r) ( 5.1.30)
"N", n=O N",

and

(k) _ _1_~
a, - ~ -'..Jak-Jh{(N",-t}i2)+-n exp
(0,tr (n+1I2)r) (5.1.31)
"N", n=O N",

Spectrum ~!k)} can be represented through spectrum ~!k-I)} as following:

a!k)=a!k-I)exp( -itr :w)+

[(-Ir ak-I-flX«N", -1)/2)+-N", -ak_l _flX«Nw-t)/2)]ex{ -;tr 2~ w)' (5.1.32)
Recursive computations according to Eq. 5.1.32 require 4(N w -1)
multiplication and 6N w - 3 addition operations for both DCT and DcST
transform in sliding window of N sampies.
Extension of these algorithms to the recursive computation of 2-D DFT,
DCT and DcST is straightforward owing to the separability of the
transforms. For instance, for DCT/DcST in the window of N w x M w sampies
obtain:

a(k,/) = a(k-I,/) ex-'- i1ll" / N

',S ',S 1'\ w
)+ tfa k+N
- -(-1 \r - a -
,I J
-lexp(- itr_r _)
k,1 2N
w
(5.1.33)

where k = k -1- fIXllN w -1)/2), 1 = 1-1- fIXUM w -1)/2).

As one can see, in the recursive algorithm for computing DCT and DcST
in sliding window described by Eq. 5.1.31, DCT and DcST are inseparable:

(5.1.34)
 222 Chapter5

and

(k)ocST
Ir,
= Imagl~(k)}=
pr (k-l)ocSTC _
Ir, I
(k-I)DCTS
Ir, I
_ A
L.1
a k S 7.' (5 • 1•35)

where

CI = COS(I&~); Cl = COS(I&_r_); SI = Sin(I&~); S7. = Sin(l&-r_);

N", lN", N", lN "'
(5.1.36)

Many applieations such as those diseussed in Chapts 8 and 9 require

eomputing only the DCT in sliding window. In these eases, eomputations of
both DCT and DeST aeeording to Eqs. 5.1.32 and 33 are redundant. This
redundaney associated with the need to eompute the DeST for reeursive

1).
eomputing the DCT ean be eliminated in an algorithm that uses DCT
spectrum found on two previous window positions ([4 To derive this
algorithm , compute DFTs A 5DCT and A SDcST of p;
Llk)DcT and p ;,
LJk)ocST}

respectively, over index k assuming that the number of signal sampIes is

N k • According to the shift theorem of the DFT, obtain for A:CT and
A~CST from Eqs. 5.1.34 and 35:

(5.1.37)

A sDcST =ADcSTC Z_ADCTS Z- AA S (5.1.38)

s I s I s l'

where S- index in the DFT domain, AAs - DFT of Aak and

Z =ex{i2A" ;.) - the spectra1 sbift factor that corresponds to signal sbift

one sampIe back. Express, using Eq. 5.1.38, A: CST through A:CT and
substitute it into Eq. 5.1.37. Then obtain:

from which it follows that

 5. METHODS OF DIGITAL FILTERING 223

A:CT ;:: 2CIA:'CTZ -{ci + si }A:,CTZZ -(CZcI - SZSI)M.Z + CzM•.

(5.1.40)

A •DCT ;::2CIADCTZ_ADCTZZ+C
. s z(M. -Ms
Z)· (5.1.41)

Finally, taking inverse OFT ofboth part ofEq. 5.1.42 and using the fact that
spectral multiplicands Z and ZZ correspond to signal shifts one and two
sampIes back, respectively, obtain for the sliding window OCT:

~k)oCT
r
;:: 2~k-I)DCT COs(._r_) _~k-l)oCT + (da - da )cos(._r_).
r N r k k-I 2N
w w
(5.1.42)

Similar recursive relationship can be derived for the sliding window OcST:

~k)oCST
r
;:: ~k-l)ocST cos(._r_) _~k-l)oCST -(da + da )Sin(.-r_)
r N r k k-I 2N
w w
(5.1.43)

Computations according to Eqs. 5.1.42 and 43 require only 2Nw

multiplication and 2Nw + 4 addition operations.

5.1.4 Border processing in digital filtering

Eq. 5.1.1 defines digital filter output signal sampIes {bk} only for
k =N h' N h+ 1, Na -1 because input signal sampIes {ak} for k < 0 are not
defined. SampIes for k < N h and k> Na -1 are not defined. Therefore in
digital filtering the number of output signal sampIes is shrinking away unless
unavailable input signal sampIes are not additionally defined by means of a
certain procedure of signal border processing.
Border processing is approximating sampIes of initial continuous signal
lost in the result of signal discretization by extrapolating available signal
sampies onto unavailable ones. Two approaches are possible to border
processing, statistical and deterministic ones.
In the statistical approach, available signal sampIes are regarded as a
realization of a random process with certain statistical characteristics such as
mean value, standard deviation, correlation function and alike. As a first
224 Chapter5

approximation, one can regard assigning to unavailable signal sampies the

mean value of the random process, for instance, zero if the process is zero
mean. This is the rationale behind assigning zeros to unavailable signal
sampies which is very frequently practiced in digital signal processing. The
zero mean assumption is however quite unnatural in image processing and
digital holography unless it is apriori known that unavailable signal sampies
are zero. This is for instance the case in antenna array processing when it is a
priori known that that there is no radiation outside antenna's aperture. Such
situation in fact assumes deterministic approach to border processing.
In the deterministic approach, border processing is designed for every
particular signal individually and implements certain requirements regarding
signal behavior. One of the most natural requirements is that of the absence
of discontinuities at signal borders. This requirement can be met by the

. _IIJhJII I 1.. . IIIJujI 1II

a) b)

11 1'l' I r1'1' ..... I --

..... fI fl!..!.Ld f'l
1111 11 Available sampies
Extrapolated sampies
c)
Figure 5-7. Three signal extrapolation methods that secure the absence ofsignal discontinuity
on the borders: a) - repeating border sampies; b) - signal mirroring; c) - interpolation between
border sampies in the assumption of signal periodical replication

the following three extrapolation methods illustrated in Fig.5-7:

- Replicating border sampies of signal {ak}of Nil sampies when extended
signal copy is obtained as:

(5.1.44)

mirroring signal border sampies:

(5.1.45)
5. METHODS OF DIGITAL FILTERING 225

interpolating, by, for instance, linear interpolation, signal sampies

between left/right border sampies assuming signal periodical replication
with a certain period Np> Na:

(5 .1.46)

These methods should be regarded as just good practical stratagems. Two

latter methods, signal mirroring and signal interpolating between border
sampies have a direct correspondence to filtering methods in the domain of
OFT and OCT discussed in Sect. 5.2. Fig. 5-8 illustrates these three
extrapolation methods applied to an image.

Original image

Extrapolation by replicating border sampie!

Extrapolation by mirroring Linear interpolation between borders

Figure 5-8. Three methods for extrapolating images without discontinuities on its borders
 226 Chapter 5

5.2 FILTERING IN TRANSFORM DOMAIN

5.2.1 DFT and cyclic convolution. Border processing

According to the convolution theorem for Discrete Fourier Transform

(Eq.4.3.30), inverse DFT of the product {ar'l,}of DFT spectra {ar} and
{'Ir} of signals {ak} and {hk} of N sampies is proportional to the cyclic
convolution of the signals:

(5.2.1)

This property can be used for computing signal cyclic convolution as

c = IDFT{DFT{a}. DFT{h}} , (5.2.2)

where a, band C are vectors of signals and their cyclic convolution,

DFT{} and IDFT{} denote operators of direct and inverse Discrete
Fourier Transforms and • is component-wise multiplication (Hadamard
product) of vectors. We will refer to this method of signal filtering as to
Jiltering in the DFT domain.
Filtering in OFT domain is a special case of Jiltering in transform
domain:

(5.2.3)

where T and T -I are direct and inverse transform.

Because of the existence of Fast Fourier Transform (FFT) algorithms for
computing direct and inverse DFTs, such an implementation of the
convolution can be advantageous to direct convolution in signal domain
described in the previous sec ti on. As it will be shown in Chapt. 6, the
number of arithmetic operations for computing DFT with the use of FFTs
grows with N as Nlog 2 N, or log2 N per signal sampie. Therefore
computational complexity of signal cyclic convolution via FFT is
O(log2 N)per signal sampie while direct signal convolution requires
O(N h) operations for a digital filter PSF of N h sampies unless recursive,
cascade or parallel recursive implementation are used. This comparison
shows that for large enough N h the cyclic convolution via FFT may become
faster then the direct convolution. This is especially true for 2-D and multi-
 5. METHODS OF DIGITAL FILTERING 227

dimensional signal for which the computational complexity of the direct

convolution grows as N': , where m is the signal dimensionality and N h is
I-D size ofthe filter PSF, while FFT complexity grows as mlog z N. The
same comparison holds for general filtering in transform domain (Eq. 5.2.3)
provided that fast transform algorithms exist. They will be reviewed in
Chapt. 6.
The cyclic periodicity of digital convolution via FFT is rather unnatural
for real life signals. It may cause substantial filtering artifacts in the vicinity
of signal borders. With cyc1ic signal replication that is assumed when the
DFT is used, signal sampies situated near opposite signal borders are
regarded as near neighbors though, in fact, they are separated by the entire
signal extent. In particular, strong discontinuities on signal borders may
cause substantial Gibbs effects in filtering. These cyclic convolution artifacts
can be reduced by means of signal extension by linear or other interpolation
between opposite signal borders described in the previous section (Eq. 5.1.46
and Fig. 5-8). A more radical solution of the problem of border artifacts is
signal mirroring which leads to the signal convolution in the DCT domain.

5.2.2 Signal convolution in the DCT domain

Let signal {ak} is obtained from signal {ak}of N sampies by its

extension by mirror reflection and periodical replication of the result with a
period of 2N sampies:

_ { ak , = 0,1,..., N -1;
k
a(k)mod2N = a - - , k = N,N = 1,•••,2N -1
. (5.2.4)
2N k 1

and let {h .. } is a convolution kerne I of N sampies ( n =0,1, ..•, N -1). Then

N-l

ck = Lh"ii(k-..
..=0
-+(N/2])mod2N (5.2.5)

where

[ NJ {N /2, forevenN (5.2.6)

2 = (N -1)/2,foroddN
is a convolution of extended, by mirror reflection, signal {ak} with kernel
{h.. }. It will coincide with the cyclic convolution of period 2N :
 228 Chapter5

N-I_

C(k)mOd2N =L
n=0
"<n)mOd2NQ(k-IH{N /2Dmod2N (5.2.7)

for kernel

n = O, ... ,[N 12]-1

n =[N 121...
,[N 12]+ N -1. (5.2.8)
[N 12]+N,...,2N-l

The cyc1ic convolution described by Eqs. 5.2.7 and 8 is illustrated in Fig. 5-

9.

11'lll; nl,ii,:'li ~.i': ~.} 11111

I

~ 1",1'1,1"
000000'

Convolution k=d {h.};n;"

0000001
~.i;on ' :
11
r: 0000000

""'rn,vOIUtiOn tenn

11 1 0000000

Convolution kernel ~.} in its position for the middle convolution term

0000001
1 I 11 1 I1 11 I 1 1 0000000

Convolution kerne I ~.} in its position for t~e last convolution term

Figure 5-9. Cyclic convolution of a signal extended by its mirror reflection from its borders

As it follows from Eq. 4.2.27 for shifted cyc1ic convolution, for such signals
of double length 2N , one can retain advantages of computing convolution
with the use of a FFT if one DFT in the Fourier domain convolution
algorithm described by Eq. 5.2.2 is replaced by SDFT(u, v) with shift
parameters u = 11 2 and v = 0 :
 5. METHODS OF DIGITAL FILTERING 229

c =ISDFT1/l,0 ~DFTI/l'O {a}. DFT~}} (5.2.9)

It was shown in Sect. 4.3.3, that for signals extended to their double
length by mirror reflection, SOFT (1/2,0) is reduced to the Discrete Cosine
Transform. Therefore Eq. 5.2.9 is equivalent to the equation:

c = ISDFT1/l,0 ~cr{a}. DFT~}}, (5.2.10)

where vector ~:CT}= Dcr{a} of OCT transform coefficients is assumed to

be extended to its double length according to its symmetry property (Eq.
4.3.59):

aDCT = _aDCT . aDCT = 0

, 2N-,' N . (5.2.11)

In virtue of this symmetry inverse SOFT ISDFTI/2 ,o for computing

convolution according to Eq. 5.2.7 is reduced to inverse OCT and OCsT
transforms:

c=
k
1 2~1 a,DCT-'I, exp['2tr (k + 11 2}rJ =
~ ~ -I
~2N ~ 2N

~
~2N
1 {~DCT-'I, ['2
~a,
,=0
exp -I tr (k+1I2}r] +
2N

La:CTii,exp[(k+1I2)r]}
2N-I
-i2tr =
,=N 2N

-1- {a:cT'Io+La: N-I CT [ 'I,exp( - i(k+1I2»)]

trr +
.J2N ,=1 N

La:CT'1: exp i (k+1I2)]}

N-I
trr = [

,=1 N

- 1- {aDCT + 2 ~ a DCT ,,'" cos tr (k+1I2)]

ft
N-I
r [
.J2N "10 f:t"
0 N

- 2~
N-I
a:CT'I;m sin[(k+1I2)]}
.
N
tr r , (5.2.12)

where ~;e} and ~~m} are real and imaginary parts of OFT coefficients {'1,}
ofinterpolation kernel {hk}:
230 Chapter5

'Ir = 1
t.; L
2N-1 - ( • krJ
h" exp ,21&- , (5.2.13)
'\IN k=O 2N

and ~;} are complex conjugates to {",}.

The computational complexity of computing signal convolution in the
OCT domain by Eq. 5.2.10 is, by the order ofmagnitude, the same as that of
the above OFT domain convolution algorithm. However fast algorithms for
OCT, IOCT and IDST exist [5-12] that require, in fact, even less
computations than FFT and IFFT involved in the OFT domain algorithm.
 5. METHODS OF DIGITAL FILTERING 231

5.3 COMBINED ALGORITHMS FOR COMPUTING

DFT AND DCT OF REAL VALUED SIGNALS

As it was already mentioned, for computing the DFT and SDFT, FFT
algorithms are used that substantially reduce the computational burden of
digital filtering. The use of FFT assumes that data are complex numbers (see
Sect. 6.1). However in many image processing and digital holography
applications data are represented by real numbers. For direct application of
FFT algorithms to such data one should treat them as complex numbers with
zero imaginary part. This leads to additional computation and memory
expenditures. The DCT deals with real numbers and there exist special fast
DCT algorithms that are described in Sect. 6.3.6. However the DCT can also
be computed with FFT algorithms in virtue of the link between the DCT and
SDFT(1I2,O). Direct use of a FFT for computing the DCT exhibits four-fold
redundancy: two-fold redundancy owing to the fact that data are real
numbers and additional two-fold redundancy owing to the signal
symmetrization. Combined algorithms described in this section enable
avoiding these unnecessary expenditures.

5.3.1 Combined algorithms for computing DFT of signals with real

sampIes

Let "'k= a:} is signal with N real sampies and {a,} is a set of its
complex DFT coefficients. According to Eq. 4.3.42, {er,} exhibit complex
conjugate symmetry

(5.3.1 )

This means that one needs to compute only half of the coefficients {er,}, the
other half can then be found from Eq. 5.3.1. However FFT algorithms are
capable of efficient computing of only all transform coefficient vector as a
whole unless more complex and less widespread "pruned" transforms (See
Sect. 6.4) are available. One can avoid this limitation and efficiently exploit
the spectrum symmetry for reducing the number of operations needed for
computing signal DFT spectrum with conventional FFT algorithms. Two
ways are available to this goal: (i) by simultaneous computing spectra of two
signals with real sampies and (H) by splitting one signal with real sampies
into two signals, computing their spectra and then combining them into the
spectrum of the initial signal.
232 Chapter5

Consider the first method. Let ~k = a;} and {bk = b;} are two signals
with N real sampies and ~, =a~_,} and {P,
=fJ~-,} are their respective
DFT spectra. For an auxiliary signal with complex sampies:

(5 .3.2)

and compute its DFT spectrum:

(5.3 .3)

Consider now

,,~-,}= ~~-, - ifJ~_,}= {a, - ifJ,} (5.3.4)

and obtain that spectra {a,} and {P,} can be found as:
(5.3.5)

(5.3.6)

Flow chart ofthis combined algorithm is shown in Fig. 5-10 a.

{cA} !?FT

a)
Re

IFFT
Im

b)

Figure 5-10. Combined algorithms for simultaneous computing direct and inverse DFT of
two signals via FFT
5. METHODS OF DIGITAL FILTERING 233

Reversion of this algorithm as it is shown in Fig. 5-10 b) gives a combined

algorithm for simultaneous inverse OFT of spectra of two signals with real
sampies.
Consider now the second method. Let {a k } is a signal with 2N real
sampies. Its OFT can be computed as

a, = r:::-:;
...;2N
L a k exp( i21r-
1 ZN - I
k=O
kr)
=
2N
1 [N-I
r;:-;:;
...;2N k=O
Lau exp(2kr)
i21r- + Lazk+1 exp(i21r--r
2N
2 k + 1 )] =
2N
N-I

k=O

1r;;; { r.-;La
1 N-I
...;2 ...;N k=O
Zk exp(i2tr-
kr) +
N

[JN ~a".. eXP(i2K~ )JeXP(iK 2~ )}~

~[OFT{alk}+exP(ilr ~}OFT{aZk+.}]' (5.3.7)

Therefore computing the OFT ofthe signal can be reduced to computing two
OFTs of two auxiliary signals formed from its even and odd sampies. The
latter can be carried out by the above-described combined algorithm.

~k =a;} {e" =:; lllJf + ia1>r+l}

k=O,1, •.. ,2N-l 1J=l}.l••••,N-l

{exp(i2nr / N)}

Figure 5-11. Flow diagram ofthe combined algorithm for computing DFT ofa signal with
even number of real sampies
234 Chapter 5

2-D OFT is usually computed as two successive passes as raw wise and
column wise 1-0 OFTs. The above combined algorithms can be used at the
first pass, and the first algorithm is weB suited for simultaneous computing
1-0 OFT ofpairs ofimage rows. At the second (column-wise) pass, data are
complex. However, in virtue of their complex conjugate symmetry,
additional saving is possible as schematic diagram of Fig. 5-12 shows.
Because 1sI and (N/2+ 1)-th columns of 2-0 array of 1-0 row spectra
obtained at the first pass, where N is the number of image columns, are real
numbers, their column-wise spectra can be computed simultaneously with
the combined algorithm. Then one needs to additionaBy compute 1-0 OFTs
of N/2-1 columns beginning from the second one.

1-----------
I'· pass: I 2nd pass:
Combined DFTs of pairs of rows 1- Combined DFT ror I" and (N12+1)-th
,columns
.... 1 r DFTs for columns #2, .•.,N/2
.......
•
I . . •••••

...
...
...
• ••••••

........
•••••
1

~I
1

J.
I
••••

.,. II I
....
...
••••••

... r14.·····
I
I • ••••.
. .lex."
/.X'lt ....
.
. .
.....
~

I -""'-
... •••••• 1
~ X

loIk,..,. ••• .. ~ )C.)(» • • • ,

I
-----------
I-D Spectra of image rows Final 2-D image spectrum
1 ______ - - - - -

•..
o
~.
-real valued sampies

Pair-wise conjugate sampies

Figure 5-12. Flow diagram of two pass 2-D DFT with the use of combined algorithms

5.3.2 Combined algorithm for computing the DCT via FFT

Though special fast algorithms for computing the OCT are available
(Sec. 6.4), it can also be computed using FFT packages that are more
widespread. This option makes use of the fact that the OCT is SOFT(1I2,O)
of signals with even symmetry:
 5. METHODS OF DIGITAL FILTERING 235

{a =a
k 2N - k -J, k = O,I,•..,N-l (5.3.8)

From symmetry properties of SOFTs (Sect. 4.3.2) it follows that, for

SOFT(1/2,0)

and

{a exp[i2n' N(k2:1I2)] =i(-IYa

k k } ~ SDFT(1/2,0)~ ~:~~o}
(5.3.10)

These relationships suggest the following method for simultaneous

computing the OCT of four signals. Let ~I,k}' ~2,k}' ~3,k}' ~4,k} are four
signals with N real sampies. Form auxiliary signals

bk =al,k + ia 2,k; ck =a3,k + ia 4,k;

- {bk'
bk =
k = 0,1,•.•, N -1
;
b2N _ k _ l , k = N,N +1, .•.,2N-l
_={c
ck
k' k = 0,1, ..•, N - 1
(5.3.11)
C 2N _ I _ k ' k=N,N+l, ...,2N-l

and

(5.3.12)

and compute SOFTe 1/2,0) of the composite signal {Zk}' Then obtain, with
an account for Eq. 5.2.10:

,!1/2,O) =SDFTII2 ,O {Zk}= SDFTI/2,O ~}+ SDFTI/2,O ~(-1 Yck}=

p!I/2,O)+,t;,o) (5.3.13)

From this and the symmetry property (Eq. 5.2.9) two equations follow:

,(1/2,0) = R(I/2,O) _ r-(1/2,O)

~r I'r N-,
,(1/2,0) = p-(1/2,o) + r-(1/2,O) :;:: _ ;'(1/2,0) _ r(I/2,O)
.., 2N-r 2N-r 3N-r Pr N-r' (5.3.14)
236 Chapter5

Therefore

j(I /2,0)
I'r
=1,(1/2,0)
\~r
_ ,(1 /2,0»)1 2 .
~2N-r , (5.3.15)

P-(1N-,/2,O) =1,(112,0)
\~,
+ ~2N-,
,(1 /2,0»)1 2 . (5.3.16)

The latter equation can be written alternatively as

j(I /2,O)
J'r
=1,(1/2,0)
\~ N+r
+ ,(112,0»)/2
~ N-r . (5.3.17)

Finally we obtain from Eqs. 5.3.15,16 and 11 that DCT spectra ofsignals
~I,k}' ~2,k}' ~3,1<}' ~4,k} can be found as:
a IDCT
,'
=DCT{a1,Je }= Ri,(1/2,O)
\-. r
_ ,(1 /2,0»)1 2 .
':t 2N-r ,

a 2" =DCT{a 2,1< }= Im\~,

DCT / ,(1I2,0) _ ,(1 /2,0»)12 .
~ 2N-' ,
a 3,,,DCT =DCT{a 3,k }= Rl, .. (1 /2,O) + ,(1/2,0»)1 2 .
\~ N+r "N-r ,

a 4" =DCT \"4,1< }= Im \~N+'

DCT L l,(1 /2,0) + ,(1 /2,0»)/2
~N-' . (5.3.18)

Block diagram ofthis combined algorithm is shown in Fig. 5-13.

ReVt} ~I,,}
Imvrl
~2,k = a 2.2N_I_k }
ReV2N_, } k~)
Imfr2N_,}
RefrN+'}
~J.k =a J.2N-I-k }
ImVN+,} kJ
{a!1 = a~~~N_I_k} RefrN-' }
Imfr _,} ~4J

Figure 5-/3. Block diagram of the combined algorithm for computing DCT of four signals
5. METHODS OF DIGITAL FILTERING 237

References
1. C. de Boor, A Practical Guide to Splines, Springer Verlag, N.Y., 1978
2. M. Unser, A. Aldroubi, "Polynomial Splines and Wavelets - A Signal Processing
Perspective, In: Wavelets - A Tutorial in Theory and Applications, C. K. Chui, Ed., San
Diego, CA, Academic, 1992, pp. 91-122
3. R. Y. Vitkus and L.P. Yaroslavsky, "Recursive Algorithms for Local Adaptive Linear
Filtration", in Mathematical Research., L.P. Yaroslavskii, A. Rosenfeld and W.
Wilhelmi, Ed., Berlin: Academy Verlag, pp. 34-39,1987.
4. Jiangtao Xi, J. F. Chicharo, Computing Running DCT's and DST's based on Their
Second Order Shift Properties, IEEE Trans. On Circuits and Systems -I: Fundamental
Theory and Applications, vol. 47, No. 5, May 2000, pp. 779-783
5. Z. Wang, "Fast algorithms for the discrete W transform and for the Discrete Fourier
Transform", IEEE Trans. ASSP, vol. ASSP-32, pp. 803-816, Aug. 1984
6. H.S. Hou, "A fast recursive algorithm for computing the Discrete Cosine Transform",
IEEE Trans. on ASSP, vol. ASSP-35, No. 1O,0ct. 1987, p. 1455-1461
7. Z. Wang, "A simple structured algorithm for the DCT, in Proc 3rd Ann. Conj Signal
Process. Xi 'an, China, Nov 1988, pp.28-31 (in Chinese).
8. A. Gupta, K. R. Rao, "A fast recursive algorithm for the discrete sine transform", IEEE
Trans. on ASSP, vol. ASSP-38, No. 3, March 1990, pp. 553-557.
9. Z. Wang, "Pruning the Fast Discrete Cosine Transform", IEEE Transactions on
communications, Vol. 39, No. 5, May 1991.
10. Z. Cvetkovic, M. V. Popovic, "New fast recursive algorithms for the computation of
Discrete Cosine and Sine Transforms", IEEE Trans. Signal Processing, vol. 40, No. 8,
August 1992, pp. 2083-2086.
11. J. Ch. Yao, eh-V. Hsu, "Further results on " New fast recursive algorithms for the
diserete eosine and sine transforms", IEEE Trans. on Signal Proeessing, vol. 42, No. 11,
Nov. 1994, pp. 3254-3255
12. L. Yaroslavsky, A. Happonen, Y. Katyi, "Signal discrete sinc-interpolation in DCT
domain: fast algorithms", SMMSP 2002, Second International Workshop on Spectral
Methods and Multirate Signal Processing, Toulouse (France), 07.09.2002 - 08.09.2002.
 Chapter 6

FAST TRANSFORMS

6.1 THE PRINCIPLE OF FAST FOURIER

TRANSFORMS

Fast Fourier Transforms (FFT) are algorithrns for fast computation of the
DFT. The principle of the FFT can easily be understood if one compares I-D
and 2-D DFT. Let ~!1)} and ~!~n be I-D and 2-D arrays with the same
number N=N\N 2 of sampIes: n=O,I,...,N\N2 -1; k=O,l, ...,N\-l;
1= O,1, •.. ,N 2 -1. Direct computing ofthe I-D DFT ofarray ~!\)}

1 N-\
a, =--La~\)exp i21r- (nr) (6.1.1)
IN n=O N

requires N 2 = N \2 N; operations with complex numbers while computing

the 2-D DFT of array ~~~n

(6.1.2)

requires only N:N 2 +N\N; =N\N 2 (N\ +N 2 ) operations as the 2-D

DFT is separable into two I-D DFTs:
 240 Chapter6

(Zr •• = 1 NI-I
ra;
"N k=O
L [
exp ;21f-
NI 1=0
La" exp ;21f-
kr ]N2-1 [ls ]
Nl
(6.1.3)

Therefore one can accelerate computing the OFT by representing it in a

separable multi-dimensional form. One can do it if the size of the array is a
composite number. Let, as in the above example, N =NIN1 • Represent
indices k and , of signal and its transform sampies as two-dimensional
ones:

k = klN I + k l ; k l = 0,1,•••, NI -1; k l = 0,1,•••, N l -1;

, = 'I N 1 +'1; 'I = 0,1, •••, NI - 1; '1 = 0,1,••. , N 1 -1; (6.1.4)

Then the 2-D OFT in Eq. 6.1.2 can be split into two successive 1-0 OFTs:

This computation scheme is illustrated in flow diagram of Fig. 6-1.

Obviously, the larger is the number of factors of N the higher is the
dimensionality to which 1-0 OFT can be decomposed in this way, and the
higher is the computational complexity reduction. For n factors of
N = No . NI',•••,. N ,,-I the OFT can be computed with only
N.(No+NI+,...,+N,,_I) operations with complex numbers instead of
Nl = N(N o ' NI·,•••,.N,,_I) operations required for the direct computation.
For N = 2" computational complexity of the OFT is O(Nlog l N). Fast
Fourier Transforms for signal size that is apower of 2 are called ,adix-2
FFT. They are most widespread in the software packages for signal
processing.
FFT algorithms are usually illustrated in the form of graphs. A typical
graph of an 8 points radix 2 FFT is shown in Fig. 6-2. Signs "-" at graph
nodes denote that values at the correspondent graph edges are subtracted.
Circ1es at the nodes signify multiplication by exponential "twiddle" factors
(see Eq. 6.1.5).
 6. FAST TRANSFORMS 241

k, =; O;k, = 0
..._---tM " =0
DFTl , . DFT.,.
k, =J '1= 0
k, =0 ' Z= 0
k, =3
" = 3
k, =3 k, = 0
" =0
k, =J;k, =0 '1= 1;
DFT.,.
el{i2.:~)
k, = 1
'1 =I
........
k, =3 " =3

•••••••• '1 =2; " =0

el{;2Jr~ )
DFTl , .
k, =1 DFT",.
k, = 3
k, =3 " =2
k, =3 --'L_--.Jr--;___.....J " =3

Figure 6-1. Flow diagram ofFFT for N = 12 = 3 x 4

a,

a.

as

a,.__. . a , __- - - f

a) b)

Figure 6-2. Flow charts of an "in place" FFT (Ieft) and ofFFT without pennutations; numbers
in circles are values ofindex m in exponential twiddle factors exp(i2mn / 8)
242 Chapter6

One can see on this graph that the computations are split into logz 8 = 3
stages and are carried out in each stage "in-place", without a need to
allocate separate computer memory storage for intermediate arrays of values.
One can also note that output spectral data are arranged in this algorithm in a
bit-reversed order (see also inverse order of two-component representation
ofsignal and its spectrum indices in Eq. 6.1.4). The bit-reversed order is the
order sequential for binary codes of indices written in reversed order of
binary digits. Such an arrangement of output data may require an additional
computational stage for "bit reversa" data reshuffling into their natural
order. Algorithms with bit-reversed order of spectral coefficients are, in a
signal processing jargon, called "decimation-in-time" FFT algorithms. By
means of left-to-right mirroring the graph a "decimation-in-frequency"
FFT graph with bit reversed ordering of input data is obtained. The data or
spectra reshuffiing can be avoided by incorporating them into the graph as it
is illustrated in the graph of Fig. 6-2, b). The FFT algorithms with natural
ordering of input and output data do not enable "in-place" computation.
FFT algorithms are a special case of a broad class of fast transform
algorithms that have similar structure and whose computational complexity
is O{N log N J or lower.
6. FAST TRANSFORMS 243

6.2 MATRIX TECHNIQUES IN FAST TRANSFORMS

Graph representation of fast algorithms illustrated in Fig. 6.2 enables

visualizing the algorithms for any particular transform size and helps to
understand the algorithm structure. Matrix techniques are compact and
general means for describing transformation and deriving fast trans form
algorithms in a way that permits automatic conversion of the algorithm
formal description into a program code and facilitates evaluation of its
computational complexity ([I]).
In matrix interpretation, signal transformations are treated as
multiplication of a vector-column composed of signal sampIes by a
transform matrix and fast transform algorithms are interpreted as
factorization of transform matrix into a product of sparse matrices.
We will describe the matrix techniques using the following denotations.
Matrices:
Matrix M V •H is a rectangle array of V x H numbers Pv.h:

Po.o Po.1 Po.H-1

PI.o PI.I
MV •H =
Pv-I.O Pv-I.H-I

Zero - matrix 0V.H is a matrix of V x H zeros;

Ones - matrix l v •H is a matrix of V x H ones;
"Bif'-matrix [10]is a vector-row [1 0];
Reverse "Bif'-matrix [Ol]is a vector-row [0 1];
Delta-matrix is a vector-row

where 60 is Kronecker delta and s =O,I"",S -1.

[11]= [10]+[Ol]=~ 1];
[11]=[10]-[01]=[1 -1];
Identity matrix I s is a square S x S matrix
1 o o
1
I
•
= 0
o
o o 1
244 Chapter6

Conjugate identity matrix r is an operator of converting vectors to

their complex conjugate: r· a =a'
Reverse identity matrix I s is a square S x S matrix:

o o 1
1 o

o o
Multiplied by a vector, it changes the order ofvector elements.
Hadamard matrix 0/ order 2:

h=[1 1].
2
1 -1

Matrix operations:
Matrix sum o/matrices:

N-I
~M(k)
~ JI,H
=
k=O

N-I (k)
LPJI-1,H-1
k=O

Matrix product oftwo o/matrices:

!:P~?p~~!
9=0

( ) .M(2)
MR,S -MI -
~ p(I,I!p~2,o)
L ..
- R,Q Q,S - 9=0

Inverse matrix M-I to matrix M is defined by the equation

6. FAST TRANSFORMS 245

Matrix transposition:

Po,o Po,I Po,H-1 " Po,o A,o /Iv-I,O

(Mv,HT = A,o A,I

=
Po,I A,I

/Iv-I,O /Iv-I,H-I Po,H-1 /Iv-I,H-I

Direct matrix sum of two matrices:

Direct matrix sum of N matrices:

Vertical (row-wise) matrix sum oftwo matrices

Vertical (row-wise) matrix sum of N matrices:

Horizontal (column-wise) matrix sum oftwo matrices:

~2) ].
"',H2 '

Horizontal (column-wise) matrix sum of N matrices:

246 Chapter 6

Kronecker product of two matrices:

Kronecker product of N matrices:

N-th Kronecker power 0/ a matrix (Kronecker product of N identical

matrices):

N-l
M8N = ®M
k=O

Kronecker matrix is matrix that is representable as a Kronecker product

of other matrices.
Layered Kronecker matrix is matrix that is representable as a vertical
sum of Kronecker matrices.

For facilitating algebraic manipulations with matrices, the following

matrix identities are useful ([1,2]):

(6.2.1)

N-l
E9 M=I ®M; (6.2.2)
k=O N

(6.2.3)
6. FAST TRANSFORMS 247

(6.2.4)

(6.2.5)

(6.2.6)

(6.2.7)

(6.2.8)

(6.2.9)

(6.2.10)

(6.2.11)

Matrix factorization identities:

(6.2.12)

( 6.2.13)
248 Chapter6

( 6.2.14)

( 6.2.15)

(6.2.17)

(6.2.18)

Matrix permutation identities:

M(l) ®M(l)
v, .G, V2.G2
=[V~t(1 V ®A(k»)~.IM(l)
k=O 2 v, J ~ V2.G2
®M(I) ).[vgl(1
v, ,G, k=O G,
®A(k»)~.
G J'
2

(6.2.20)

(6.2.21)

(6.2.22)
 6. FAST TRANSFORMS 249

6.3 TRANSFORMS AND THEIR FAST

ALGORITHMS IN MATRIX REPRESENTATION

In this section we will provide matrix formulas for orthogonal transforms

listed in Table 3.1 and some other useful transforms and for their
corresponding fast algorithms beginning with simplest "binary" transforms.

6.3.1 Walsh-Hadamard transform

Walsh-Hadamard transform matrix of order r is representable as a n -th

Kronecker power of order 2 Hadamard matrix h 2 :

(6.3.1)

It is obviously representable also in a recursive form as:

and in a layered Kronecker matrix form as:

s[
HAD 2" = [11fn ~:~llrn-r) ® (11]® h~(r-') = ll]
[11fn s[~:kllrn-r) ® (11]® HAD 2,-111] . (6.3.3)

Using matrix factorization identity of Eq. 6.2.15, one can straightforwardly

factorize Hadamard transform matrix into a product of two sparse matrices:

(6.3.4)

Continuing the factorization recursively, obtain finally:

HAD 2" = n[I 2----1 ®h 2®IJ=

r=O

(6.3.5)
250 Chapter6

Eq. 6.3.5 is a matrix formula for fast Hadamard transform. It can be

directly translated into a program code. Graph of 8 point fast Walsh-
Hadamard transform is shown in Fig. 6-3, a).

ao t.ro ao (10

a\ (1\ a\ (1\

a2 (12 a2 ~

a3 (13 a3 (13

a4 (14 a4 (14

as (15 as (15

a6 (16 a6 (16

a7 (17 a7
a) b)

Figure 6-3. Flow diagrams of Fast Hadamard (a) and Fast Haar (b)
Transform algorithms.

One can observe the isomorphism between Eq. 6.3.5 and the graph
representation ofthe fast Walsh-Hadamard transform shown in Fig. 6-3, (a).
Term r = 0 in Eq. 6.3.5 corresponds to the right most stage in the graph.
Term r = n -1 corresponds to the left most stage in the graph. Term
described by matrix h 2 is represented in the graph by the "butterfly" uni!
that computes sum and difference of two inputs.
Eq. 6.3.5 explicitly shows also that Walsh-Hadamard transform of a
vector of N = 2" sampies can be computed in n = log2 N steps with
N additionlsubtraction operations in each step, or, totally, with
N log2 N additions/subtractions.

6.3.2 Walsh-Paley Transform and Bit Reversal Permutation

Walsh-Paley Transform matrix is obtained from Walsh-Hadamard

Transform matrix by means of bit reversal permutation of its rows ([3]):
 6. FAST TRANSFORMS 251

(6.3.6)

where BRM zn is a bit-reversal permutation matrix. Walsh-Paley Transform

matrix ean be represented in a reeursive form as

(6.3.7)

Using Eq. 6.3.7 reeursively yields representation of Walsh-Paley transform

matrix in the form of a layered Kroneeker matrix as

(6.3.8)

One ean eompare Eqs. 6.3.7 and 6.3.8 and Eqs. 6.3.2 and 6.3.3 to see that
bit reversal permutation that eonverts Walsh-Hadamard transform matrix
into Walsh-Paley matrix ean be attributed to permutation of their
eorresponding eonstituent elementary matriees.
Bit reversal permutation matrix, in its turn, ean be represented in layered
Kroneeker matrix and reeursive forms as

(6.3.9, a)

(6.3.9, b)

As one ean see, it is isomorphie to the Walsh-Paley matrix with substitutions

[10]~[11] and [Ol]~[fi].
Walsh-Paley transform matrix ean be faetorized using faetorization
identities (6.2.14-16) to obtain the following matrix formulas forJast Walsh-
Paley transJorm:

PAL zn = fi {Izn-,-. ®[(I z, ®[ll~(lz' ®[11)]}.

,=0
(6.3.10, a)

and, from Eqs. 6.3.5 and 6.3.6,

 252 Chapter 6

PAL 2n = BRP2n • TI {1
r=O
2n -,-t ® h 2 ® 12, }. (6.3.10, b)

In virtue of the isomorphism between Walsh-Paley and bit revers al

permutation matrices, one can also obtain a matrix formula for fast bit
revers al permutation:

(6.3.11)

from which it follows that bit reversal permutation of a vector of N = r

sampies can be carried out with N log2 N permutations of two sampies (n
steps with N permutations in each).
Note that bit reversal permutation is a special case ofmatrix transposition.
Matrix transposition can, in a similar way, be implemented through
transpositions of small sub-matrices and subsequent transposition of the
order of sub-matrices ([ 1)).

6.3.3 Walsh Transform and Binary-to-Gray Code and Hadamard

Permutations

To introduce Walsh transform matrix, we need a not ion of Gray code for
binary representation of numbers. m -th binary digit ~~c of Gray code of a
number written in its direct binary code representation as {-r-:BC },

m = O,I,••• ,n -1 is computed as

~GC
m
=~DBC
m
+ ~DBC
bin m+1
(6.3.12)

where m =0 corresponds to the least significant digit and lt,,} denotes

modulo 2 addition. An illustrative example of binary code to Gray code
conversion is given in Tab. 6-l. For comparison, the table contains also bit
reversal binary codes.

Table 6-1. Binary and Gray code representation of8 first natural numbers
Decimal index 0 1 2 3 4 5 6 7
Direct binary code 000 001 010 011 100 101 110 111
Bit reversal code 000 100 010 110 001 101 011 111
Gray code 000 001 011 010 110 111 101 100

Walsh transform matrix is obtained from Walsh-Paley matrix by means of

permutation ofits rows according to Gray codes oftheir index ([3)):
 6. FAST TRANSFORMS 253

(6.3.13)

where BGCP2" is a direct binary-to-Gray code permutation matrix. The

combination BGCP"
2
. BRP2" of direct binary-to-Gray code permutation and
bit reversal permutation converts Hadamard transform matrix into Walsh
transform matrix of which rows are arranged according to the number of
zero crossing in the sequence of elements of their rows (sequency, see Sect.
3.2.2.2)
Walsh transform matrix may be represented in both recursive form:

(6.3.14)

and a layered Kronecker matrix form as:

Comparison of Eqs. 6.3.14 and 6.3.15 with corresponding Eqs. 6.3.7 and
6.3.8 shows that direct binary-to-Gray code permutation is associated with
reverse identity matrix 12 .
Matrix HADP" 2
= BGCP2" . BRP"2 that connects Walsh transfonn and
Walsh-Hadamard Transform is called Hadamard permutation matrix ([4]).
It is isomorphie to the Walsh transform matrix with substitutions
[11]~[10] and [lr]~[10]. With these substitutions obtain from Eq.
6.3.14:

(6.3.16, a)

and from Eq. 6.3.15:

HADPZ" =[10 r ng {{ §[(iZ'-1 .HADP ,_I)®[lrB®[11r<n-r)}}.

2

(6.3.16, b)

By inversion ofthe order ofKronecker products in Eq. 6.3.16, b), one can
separate from the product HADPZ" = BGCPZ" • BRPZ" the direct binary-to-
Gray code pennutation matrix:
254 Chapter6

BGCP2_=[10rn E{{ ~[11r<n-r)®(tl]®(12'_1 .BGCP2,_1 .BRP2 ,-J}}

(6.3.17)

Applying recursively factorization identities (Eqs. 6.2.11-18), one can

obtain a matrix formula for Fast Walsh Transform:

WAL 2_= {Ü {12 n-z-, ®[~2'1 ®[11]}3(1 2, ®[11])]e

IZ'+I .[(12'1 ®[ll]}s(1 2, ®[11])].lz,+I}}X (6.3.18)

[(1 2--1®[ll]}3lz_-1(1 2,-1 ®~1])]

Substitutions [11]~[10] and [11]~[10] convert this formula into the
formula for fast Hadamard permutation:

HADP2" ={g {12_-Z-, ®[([(12'1 ®[10]}3(12, ®[Ol])])]e

IZ'+I .[(12,1 ®[1O]}E(12, ®pl])].lz,+I}}X (6.3.19)
[[(12-_1®[10~iZ__1(12,-1 ®[Ol])])]
Matrix formula for fast direct binary-to-Gray code permutation can be
obtained from the relationship of Eq. 6.3.13 between WAL and PAL
matrices as ([5]):

BGCP2_=fi[1 2, ®(1 2-_,-1 ei2_-,-ll (6.3.20)

r=O

6.3.4 Haar Transform

Haar transform matrix can also be represented both in a recursive form:

(6.3.21)

and as a layered Kronecker matrix:

 6. FAST TRANSFORMS 255

(6.3.21)

Similarity between Haar transform matrix as described by Eq. 6.3.21 and

layered Kronecker matrix representation of Walsh-Paley transform (Eq.
6.3.8) is obvious. Recursive factorization of Eq. 6.3.20 yields the following
matrix formula for Fast Haar Trans/orm:

(6.3.22)

Basis functions of Haar transform in its matrix representations of Eqs.

6.3.20-23 are not normalized to be orthonormal. With the normalization, fast
Haar transform can be written as ([5]):

=(
HAR I" lEB !l(r-,,)tIII')O {(lI' ®[ll]}E(II, ®[ll])EBII"_I,+t} .
(6.3.23)

6.3.5 Discrete Fourier Transform

Matrix of the DFT of order N = 2" is, to the accuracy of the

1/
normalization factor JN the matrix

Represent index r of row elements of this matrix via binary digits {r

s }'

s = 0,1,..., n -1 of their binary code:

L
,,-1
r = rs 2 s (6.3.25)
s=O

Then matrix FOUR I " can be written in a layered Kronecker matrix form as:

(6.3.26)
256 Chapter 6

In this representation, matrix FOUR 2, contains N =r layers. One can

convert it into a recursive layered Kronecker representation with n layers by
means of the following algebraic matrix manipulations. First, represent index
k in the vertical matrix sum in Eq. 6.3.26 through binary digits {kJ,
m = 0,1, •••, n -1 of its binary code representation:

(6.3.27)

and rearrange vertical sum in Eq. 6.3.26 into n sums grouped according
{km}:

FOUR,
2
= 2'-I(n-l~
E ®
k=O .=0

(6.3.28)

Change now the order of summation over binary code digits {kJ of kin
Eq.6.3.27 to the reverse one while keeping indexing of {km} that
corresponds to Eq. 6.3.27. This will require introducing into Eq. 6.3.28 a bit
reversal matrix:

.
lJJJ}
I I ln-I 2' ~> 2"
FOUR ,
2
= BRP2" E E...
kO=O[ k, =0 [
E ® 1
k,_1 =0[ .=0[
Wn m.O
,-I
• (6.3.29)
{

Introduce, for the convenience, the bit reversed DFT matrix

FOUR 2, = BRP2, . FOUR 2, :

(6.3.30)

Rewrite Eq. 6.3.30 as

 6. FAST TRANSFORMS 257

(6.3.31)

[1 .,:- %""l [1 I] =[11] { 1 w:" %""-'' -'1 =[1 -1]= ~ I]

(6.3.32)

Now, applying matrix identity Eq. 6.2.6, one can rewrite Eq. 6.3.31 as
258 Chapter6

{ [I (I
[11]® ~ ...
kt-O
E_ [n_z[
® 1 wn_~=o
kn_t-O.=o
2
s
~km+12m ]JJ] E

[lI]®[ S...( S ['®[1

kt=O k._t=O .=0
W:'%,k Z ][10
m t m
+ ~']JJ]}
Wn
(6.3.33)

and, using identities (6.2.4) and (6.2.7), as

FOURzn = { [I (I
[11]® ~ ...
kt-O
E_ [n_z[
k._t-O
~ 1 wn_~=o
2$
.-0
~km+12'" ]JJ] E

[lI]®l~ ...( S_[[~[1 w:~~k. +tzm]J.(~[1 ~']J1]j}

kt-O k._t-O .-0 .-0 0 Wn
=

{ [ 1[I (n_2[
[11]® E ...
kt =0
E ® 1 wn_r=o
k._ t =0 .=0
2$ ~km+12m ]JJ1 E

[11-] [Ik~O (Ikn~=O[n_z[

® •••
2' I:km+t2m ]JJ1 (n-Z[1
! wn_~..o 1 ! • 0 0
w~' ]J} • (6.3.34)

Finally, using Eq. 6.3.30, obtain:

(6.3.35)

where

du =[1o exp(i2mt)J
0 1. (6.3.36)
 6. FAST TRANSFORMS 259

This formula can be recursively converted into a layered Kronecker matrix

representation with n layers as follows:

FOUR,· ~ [11t"S {~,[[11r-·) ®( ~ I]® [FOUR,.' -(~ d,•• )])]}

(6.3.37)

Derive now a matrix formula for Fast Fourier Transform. Applying matrix
identities (6.2.1,6.2.7 and 6.2.18) to Eq. 6.3.34, obtain:

FOUR2n =

(FOUR2n-1 E9FOUR2,-1 (~d2'-' )} [Ull]®1 2n-I)=:([lI]® 12n-I)]=

~OUR2n-1 E9FOUR2,-1 ){1 20 -1 E9(~d2H )}K[11Je:[lI])®1 2o-I]= .

(12® FOURz,-1 ). ( 120-1E9 (~d2'-' )). ~2 ® 120-1)

(6.3.38)

Applying this relationship recursively, obtain

FOUR2o =

fi ( 12(0-1-,) ® [1 2, E9 (! d2H2-0)]). (12(n-'-I) ® h2® 12, ) ,

(6.3.39)

or, finally,

a matrix formula for the "decimation-in-time" Fast Fourier Transform

algorithm described in Sect. 6.1 and illustrated in Fig. 6-2 (a).
Using similar matrix manipulations based on the above matrix identities
one can design many other modifications of FFT ([5,6]). For instance,
transposing matrices ofEq. 6.3.39 and using matrix symmetry yields an FFT
260 Chapter 6

algorithm with bit reversal in signal domain ("deeimation in frequency"

algorithm):

FOUR 2" =
(6.3.41)
{Ü(l y ® h, ®I,t..-J(I y ®[ I,(.~,) af~' d.~ )])}. BRP,..

It is very instruetive to eompare formulas (6.3.39) and (6.3.40) for Fast

Fourier Transform algorithms with formula (6.3.10, b) for Fast Walsh-
Paley transform algorithm. One ean see that Fast Fourier transform
algorithm ean be obtained from Fast Paley-Hadamard transform algorithm
by appending eaeh r -th of n faetor matriees of Walsh-Paley faetorized
matrix with a eorresponding diagonal "twiddle" matrix

( 12(0-,-1) ® [ 12, E9 (! d 2-(s+z) )]) . This adds, in eaeh r -th eomputation stage,

(2' -1)operations of multiplieation of eomplex numbers. Therefore total

number of operations required for eomputing the DFT with FFT algorithm
ean be evaluated as f (2' -1)= N -1- n
,=0
of multiplieations of eomplex
number plus N log N of additions of eomplex number, or totally
4(N -1- n) multiplieations and 2(N log N + N -1- n) additions of real
numbers.

6.3.6 DCT and DeST

Matriees of the DCT and DeST defined in Seet. 4.3.3 are, for N =r,
real and imaginary parts of the matrix

(6.3.42)

Applying this transform to a signal veetor with real eomponents gives a

speetral veetor whose real part is DCT and imaginary part is DeST speetrum
of the signal.
Similarly to matrix FOUR of the DFT that exhibits reeursive symmetry
when its rows are rearranged in bit reversal order, matrix DCST reveals its
symmetry when its eolumns are subjeeted to Hadamard permutation. For
DCST2 introduee the eorresponding auxiliary matrix DCST 2" :
0 ,
 6. FAST TRANSFORMS 261

DCST 2- = DCST2_HADP2_ . (6.3.43)

It was shown in [7] that Hadamard permuted matrix DCST 2- can be

represented in a recursive form via an auxiliary matrix DCST~:!s ,as:

(6.3.44)

(6.3.45)

R~~)s = ([10]® 12_- S- 1 Ja

([0 ES (- 1 I_) 1;_-s-JIJ(de .)ES 2d .1 -IJ) (6.3.46)
2_- S - s •m s •m 2--S

where s = O,I, ... ,n, m = 0,1, .•. ,2 s -1, DCST~"" = 1, I~ is the conjugation
operator,

(6.3.47)

(.) is an operator of Hadamard multiplication of complex numbers:

(6.3.48)

and m 2H1 is the position of index m in the vector of 2 s +1 indices after

Hadamard permutation of this vector.
In order to represent the fast algorithm as a product of sparse matrices, the
two inner DCST matrices in Eq. 6.3.35 should be recursively substituted.
Carrying out this substitution until the last stage obtain a direct formula:

(6.3.49)
262 Chapter6

Fig. 6-4 illustrates the matrices set for DCST of 8 points.

For input signal vector a with real elements, algorithm in this
formulation assumes that the signal has to be represented with complex
numbers as:

(6.3.50)

Then DCT and DcST transforms are obtained as:

DCST.
2
·(i = DCST2.HADP.·(i
2
=DCT.2 ·a+iDCST.·a
2
(6.3.51)

Sta~ s=o Sta~s=l Sta~s=2

r \f \f \

C) = 0 • = 1 • = -1 !Q = Real valued coefficient

>Z! = ConjOp x = -ConjOp i;i = H.+1b

Figure 6·4. Sparse matrix structure for fast DCST

Eq. 6.3.49 can be split up into formulas for separate computation of the
DCT and DcST as:

(6.3.52)

and

_
DST.
2
= DST2.HADP2• = n(2'.1
n-I

r=O
EI)
r=O
(, ))(
Rs 2:.: •.1 I,2 ® h 2 ® 12•., . 1 ) (6.3.53)
,

where

RC~:}, =([10]® 12,., ., ) a

([0 EI) (- 1
2" , . , . ) I:..,...Jm[d s •m Ea 2d•.m .1 2..,•• ..]),(6.3.54)
 6. FAST TRANSFORMS 263

and

Rs(m)
2"-.1
=
[(0 E9 I 2n-,-1 )E(es~ E9 (- 12n -'-1 ))] m(I E9 02n->-1 )E(O E9 2d
s •m I 2n->-I)]'
(6.3.55)

Fig. 6-5 shows flowchart of the DCT and DcST for N=8.
To evaluate the computational complexity of the algorithrn, denote the
number of complex multiplications of matrix M by ,u{M). Then obtain:

.u(DCST N )= ,u{R N )+ 2p(DCSTT). (6.3.56)

Considering that the input vector for each stage consists of complex
numbers, the number of multiplications of real number can be found as

r(DCST N )= 2y(R )+ 2r(DCST )= N +2r(DCST

N N /2 N ). (6.3.57)

For matrix transform of size N = 2", there are n stages in the algorithm,
and the total number of muItiplications is then N IOg2 N .

ao ao ao ag
al al al (Z I

a2 (Z2 a2 (Z2

al (Z) a) (Zl

a4 (Z4 a4 (Z4

as as (Zs

a6 ~

a7 a7 (Z7

a) b)

Figure 6-5. Illustrative flow diagram ofthe DCT (a) and DcST (b) for N=8. Solid Iines denote
addition at the graph nodes; dotted lines denote subtraction; circle denotes multiplication by
respective coefficients {ds.m} and {es~} (numerical values are not shown)
264 Chapter 6

6.4 PRUNED ALGORITHMS

In digital holography it often happens that either initial signal array

contains many zeros or else not every sampie of the signal spectrum is to be
calculated or both. The latter case is typical, for instance, for digital
reconstruction of holograms when one need to reconstruct only a fragment of
the output image plane with the object of interest. In such cases pruned FFT
algorithms can be constructed with lesser number of operations as compared
with fuH FFT algorithms ([5]). For this purpose it suffices to multiply the
transformation matrix in its factorized form on the left and on the right by
diagonal "cutting" matrices containing zeros at those points of the diagonal
whose positions correspond to the positions of zero-valued or unneeded
elements.
If the number of nonzero signal sampies and of the required spectrum
sampies is equal to apower of two, the pruning matrices are of Kronecker
type built from elementary matrices:

P=[1000]
2 (6.4.1)

and identity matrix 1 2 • The pruning matrix P:. of dimension r containing

exactly 2' ones at the beginning of the main diagonal is expressed as
folIows:

(6.4.2)

Let ones need to compute first 2k of r transform coefficients of a signal

with only 2' nonzero sampies at the beginning of the signal vector. Then
transform matrix should be truncated by multiplying it from both left and
right sides as:

(6.4.3)

Factorized representations of such a pruned FFT obtained by substituting of

Eq. 6.4.2 into Eq. 6.4.3 and using the factorized DFT matrix of Eq. 6.3.40
("decimation in frequency" algorithm) are as foHows ([5]):

FOUR(k,l)
2"
={"rr-
I
- k/I
~ 2'
® 501 ® pi2,,-I-r ).
,=0
6. FAST TRANSFORMS 265

[1 2, ®(P;n_I_,E P;n-I-, . ~ d 2+_'-I) J]}. tt(1 2, ®h2®12n-I-,)

[1 12, ® ( 2n-I-,E ~ d 2-{n_,_I»)]}' (1 21 ® S~8(n-I»)' BRP2n ; k + I ~ n. (6.4.4)

and, for k + I ~ n,

FOUR(k,/)
n
= {nn-
2
I kll
- ® SO ® pi
n ~ 2' 2 2 - 1- r
)x
,=0

[12, ® (P;n_I_,E P;n-I-, . ~ d 2+_'_I»)]}' (1 21 ® p:<n-k-/) ® S~s.)· BRP2n ,

(6.4.5)

where

(6.4.6)

and M" is transposed matrix M.

When the disposition and the number of signal zero sampies and of
unnecessary spectral sampies is arbitrary, explicit matrix formulas for
pruned algorithms are described in [9].
For the fast algorithm

FOUl\n =BR~n . fi(1 2n-I-, ®[12, E9(~d_2'-I-')]}(12n_I_, ®h 2®IJ=

n-I

B·nD, ·H" (6.4.7)

,,,,,ß

where

B= BR~n ; D, =(12n-I-, ®[12, E9 (~d _2,-1-, )]); H, =(1 2n-I-, ®h2®12, )

(6.4.8)
266 Chapter6

and left and right pruning matrices p~) and p~) containing ones in the
positions that correspond to nonzero input sampies and to transform
coefficients to be computed, pruned trans form may be written as

PFOUR2" =P(L).{B·rrn-lD
2" ,
·Hr }.p(R)=ö·rrn-l
2"
ö ·H
, T ,
(6.4.9)
r=O r=O

where

(6.4.10)

(6.4.11)
and

(6.4.12)

are stage pruned matrices and matrices p~)(B), P~)(B), P~)(Dr)'

P~R)(D,), P~)(H,), and P~)(Hr) are defined recursively using the
following notions of "reflected" cutting matrices RP(L)(M, p) and
RP(R)(P,M). The "reflected" cutting matrices are defined as:

(6.4.13)

(6.4.14)

where Sh(M) is a binary "shadow" matrix of matrix M elements of which

are ones wherever elements ofmatrix Mare nonzero.
Using the "reflected" cutting matrices of Eqs. 6.4.13-14, matrices B,
jj and ii may be found recursively as following:
p(L)/B)
\1
=p(L).
2n ,
6. FAST TRANSFORMS 267

p(L)/D
\;0
)= RP(R){p(L)
~2."
B)·
P(L)(H o)= RP(R)~(L)(Do~Do);

P(L)(D I)= RP(R)~(L)(Ho~Ho);

P(L)(HJ= RP(R)~(L)(DI ~DJ

p(L)(O"_I)= RP(R)~(L)(Hn-2~H"_2);

P(L)(H"_I)= RP(R)~(L)(On-I~O,,_J;

p(L)(O"-1)= RP(R)~(L)(Hn-2~H"_2);

P(L)(H"_I)= RP(R)~(L)(O"_I~O,,_J;

and

P(R)(H ) = peR) .
n-I 2'"

P(R)(D"_I)= P(L)(H"_I ,P(R)(H"_I »);

P(R)(H"_2)= p(L)(On-1 ,p(R)(O"_1 »);

p(R)(0"_2)= P(L)(H"_2,P(R)(H"_2»);

P(R)(Hn-3)= p(L)(O"_l,P(R)(O"_l»);

p(R)(0"_3)= P(L)(H"_3,P(R)(H"_3»);

P(R)(Ho)= p(L)(OI,P(R)(OI»);

p(R)(OO)= P(L)(Ho,p(R)(Ho»);
268 Chapter6

Fig. 6-6 shows an example of a flow diagram of pruned FFT for computing 9
spectral coefficients of signals with 7 nonzero input sampies.

-
q -
q

-<3......
L()
L()
~

c
~

Figure 6-6. Illustrative flow diagram of a pruned DFT for N= 16. Solid
lines denote multiplication by 1, or j, doted lines denote multiplication by -1,
or -j and solid lines with black bubbles denote multiplication by "twiddle"
factors.
 6. FAST TRANSFORMS 269

6.5 QUANTIZED DFT

Modifying the multiplier matriees in the matrix representation of fast

algorithms one ean generate many transforms with fast algorithms with some
special features. The Quantized Discrete Fourier Transform (QDFT) is an
example of a transform designed in this way. The QDFT approximates the
Diserete Fourier Transform but requires less or no multiplieation operations
and lesser number of addition operations.
Consider a FFT algorithm featuring bit reversal of the result of
transformation. (Eq. 6.3.39, Fig. 6-2, a).

FOUR 2• =
BRP2' . fi[ 12(.-1-,) ® [1 2,Ea (! d 2-(s+2))]} (1 2
(.-,-1) ® h 2® 12,) .
(6.5.1)

Eaeh r -th eonstituent matrix of this algorithm

(6.5.2)

is a diagonal matrix that consists of ones and twiddle factors exP(i8r ) in its
diagonal. Multiplication of eomplex numbers by these twiddle faetors may
be represented as multiplieation of a veetor eomposed of real and imaginary
parts ofthe numbers by a matrix:

TF = [ cos(Jr sin(Jr] . (6.5.3)

r _ sin(Jr cos(Jr

These multiplieations of eomplex number and eomputing eosine and sine

funetions are the most time eonsuming operations in the algorithm. To save
time for eomputing eosine and sine funetions, one of these funetions is
sometimes eomputed in advanee for all needed arguments and stored in a
look-up table. This look up table is then applied in all stages ofthe algorithm
for findin~ values of both funetions using the re1ationship
sin (Jr = cos{1r / 2 - (JJ.
Replaee eosine and sine funetions cos(Jr and sin(Jr in these twiddle
matrices by quantized funetions qos 8r and qin (Jr = qos(1r /2 - (Jr) that take
limited number of quantized values as it is illustrated in Fig. 6-7. Thus, with
270 Chapter6

three levels of quantization, qoS(lr ) and qin(lr ) take the values (-1, and °
I); with five levels of quantization, the values (-I, -112, 0, 112, I), and so on.

3 qUBQUZ8t1on 19"""5

o_S

o - - '. - I-l-"'r-r-.

-o.S

Figure 6-7. eosine function (dashed line) and corresponding qOS(.)functions for 3 and 5
quantization levels

This corresponds to replacing matrices {TF,} ofEq.(6.5.3) with the matrices

QTFr =[ qO~'r
-qmlr
qinlr ]
qoslr
•
(6.5.4)

Respectively, diagonal constituent matrices {D,} of the algorithm of Eq.

{TF,} are replaced with diagonal matrices
6.5.1 that involve twiddle factors
{QDr } with quantized twiddle factors {QTFr } on the corresponding places.
Since, by the above definition, qin(I,)= qos(.. /2 - I,) sum
(qin 20+qos20) never equals to zero though it may not always be equal to
one, matrices {QTFr } are orthogonal though not necessarily orthonormal.
To secure the proper normalization, introduce orthonormal quantized twiddle
factor matrices:

qin8r ] •
(6.5.5)
qos8r

and corresponding diagonal matrices {QD r }. In this we arrive at the

orthonormal transform
 6. FAST TRANSFORMS 271

FOUR 2n =BRP2n · nQD, . (I~n_'_I) ®h 2 ®I 2,) (6.5.6)

,=0

that is referred to as Quantized DFT.

With three quantization levels (-I, 0, I), the QOFT requires a lesser
number of addition-subtraction operations than the OFT and does not require
multiplication operation. Reduction in the number of addition-subtraction
operations is roughly inverse to the fraction of the period of qos(.)and
qin(.) in which they take zero values. With five levels of quantization (-I, -
1/2, 0, 1/2, I), multiplications can be replaced with the shift operations, and
some addition-subtraction operations are omitted too. Therefore, in its
computational complexity, the QOFT is between the Haar and the Walsh-
Hadamard-Paly fast transforms; that is, it may require considerably fewer
operations than FFT, while owing to its origin it can be used as an
approximation ofthe OFT.
Graphs in Fig. 6-8 show for comparison four basis functions of the OFT
and QOFT with 5 quantization levels for N=32.

":f1'I1~----UIL~h~~
0 .' -
·:/fI·:· ~Ir .'~I~:' Ilr': 1
--- I - ..1W ~w ..~I . JW
I ~ ,~ I'~~ 2~' 2'~ 3~ .0 .,
" ", " >0

Functions #4 Functions #5

~ ~J~N~~,~~~'~l
. .. .. ~!~W~~~I~~'~~
•0 '0

Functions #6
>0 :} 1 t') ':!J

Functions #7
2C) 2$ 30

Figure 6-8. Basis functions ofthe DFT (linearly interpolated, solid Iines) and the QDFT with
5 quantization levels (bars) for N=32

Values ofbasis functions ofthe QOFT are coarsely quantized because of

quantization of eosine and sine functions in the transform factor matrices.
However, the number of their quantization levels is greater than that of
qos(.) and qin(.) owing to interrnixing in the transform stages. Moreover, it
increases with the increasing number of the transforrn stages, that is with the
size of the transforrn matrix.
The accuracy with which rows of the QOFT represent those of the OFT
depends on selected number of quantization levels and on position of
272 Chap/er 6

quantization intervals. The latter can be optimized to minimize the

approximation error.
The QOFT was first introduced for synthesis of computer generated
holograms [6]. Its use instead of OFT was justified there by the fact that
speckle noise which inevitably appears in images reconstructed from the
holograms may hide the quantization effects of the QOFT. This is illustrated
in Fig. 6-9 that shows images optically reconstructed from digital holograms
synthesized using the QOFT instead ofthe OFT.

a) b)

Figure 6-9. Images optically reconstructed from digital holograms synthesized with the use of
the QOFT replacing the OFT: a) - the QOFT with three quantization levels; b) the QOFT with
5 quantization levels (adopted from [6])

0.8

0.6
, , .
.. , ...... r .... -,'" .... r ....

.............................
I I I
.
•

I • I ,
0,4 .. .. . . . . . . . . . . . . . . . . . . . . ,. .. ..
• I • • I

0.2 ) 1..-; - - - ~ - - -; ... ~ ..

°o 0.2 0.4 0.6 0.8

A sinusoidal signal OFT spectrum ofthe signal

0.8 ..... .. .. r .... "," .... t ...... 0.8 "',- .... ,. ..... , ..... t' ....

. . . . .. . .. . . .. .. . . . .
• t t ,
...............................
. .
• • f t
0.•

. ... . . .... .. .. . .. . . . .. .. .
0-6 I I • , • • • I

0.' 0.4 .. .............................

• I I ,

I • I I • I t I

0 .2 .. ' ....... 1 ............. j .... ..

QOFT spectrum, 3 quantization levels QOFT spectrum, 5 quantization levels

Figure 6-10. OFT and QOFT spectra ofa sinusoidal signal

6. FAST TRANSFORMS 273

Among other applications of the QOFT as approximations to the OFT,

one can mention spectral estimation, fringe analysis and generalized filtering
in the transform domain. In the generalized filtering described in eh. 8,
sampies of signal spectrum obtained by direct transform are multiplied by
coefficients characterizing the filter in the selected basis, and filtered signal
is reconstructed by inverse transform. Because normalizing coefficients of
the QOFT basis functions can be combined with the filter coefficients, no
additional normalization operations are required for the direct and inverse
QDFTs in this case. An illustrative example of spectral analysis of a
sinusoidal signal with the use ofthe DFT and QDFT is shown in Fig. 6-10.
274 Chapter 6

REFERENCES
1. L. Yaroslavsky, M. Eden, Fundamentals ofDigital Opties, Birkhauser, Boston, 1996
2. I. J. Good, J. Roy, Statist. Soe. Sero B 20, 362,1958
3. A. M. Trakhtman, V. A. Trakhtman, Fundamentals of the Theory of Diserete Signals
Defined ofFinite Intervals, Sov. Radio, Moseow, 1975, (In Russian)
4. N. Ahmed, K. R. Rao, Orthogonal Transforms for Digital Signal Proeessing, Springer
Verlag, 1975
5. L. Yaroslavsky, Digital Pieture Proeessing, An Introduetion, Springer Verlag,
Heidelberg, 1985
6. L. Yaroslavskii, N. Merzlyakov, Methods of Digital Holography, Plenum Press, N.Y.,
1980
7. L. Yaroslavsky, A. Happonen, Y.Katiyi, Signal Diserete Signal Sine-interpolation in
DCT Domain: Fast Algorithms, in : Proeeedings of International TISCP workshop on
Speetral Methods and Multirate Signal Proeessing, SMMSP'02, Toulouse ~ Franee,
07.09.2002 - 08.09.2002, Ed. T Saramaki, K. Egiazarian &1. Astola, p. 179-185, TICSP
Series #17, ISBN 952-15-08881-7, ISSN 1456-2774, TTKK Monistamo 2002
8. Y. Katiyi, L. Yaroslavsky, VIHS Strueture for Transforms and their Fast Algorithms, 3rd
Int. Symposium, Image and Signal Proeessing and Analysis, Sept. 18-20, 2003, Rome,
Italy
9. Y. Katiyi, L. Yaroslavsky, Regular Matrix Methods for Synthesis of Fast Transforms:
General Pruned and Integer-to-Integer Transforms, in Proe. Of International Workshop
on Speetral Methods and Multirate Signal Proeessing, ed. By T. Saramäki, K. Egiazarian
& J. Astola, SMMSP'2001, Pula, Croatia, June 16-18,2001
 Chapter 7

STATISTICAL COMPUTATION METHODS AND

ALGORITHMS

7.1 MEASURING SIGNAL STATISTICAL

CHARACTERISTICS

7.1.1 Measuring probability distribution and its moments and order

statistics

By definition, the distribution function P(V) of a random variable v is

the probability that the random variable does not exceed the value V. The
derivative of P(V) with respect to V

is called the probability distribution density of the random variable v.

Digital signals are characterized by discrete analogs of the distribution
function and distribution density respectively - the relative share R(m) of
the sampIes that do not exceed the given quantized value q, and the rate
h(q) of the sampIes having the value q. The latter characteristic is referred
to as signal distribution histogram, the former one as cumulative
distribution histogram.
276 Chapter 7

A histogram describing the appearance rate of quantized values of signal

sampies is called a one-dimensional histogram. A histogram that
characterizes the rate of simultaneous appearance of groups of signal values
or, for multi-component signals, groups of signal sampIe components is
called multidimensional, or a multivariable distribution histogram. For
instance, distribution histogram of three (R,G,B) components of color
images is three-dimensional histogram.
The following simple algorithm is available for computing the N-
dimensional distribution histogram h(ijN ) of N groups of M sampIes or
components of signal a = ~ im) }, k = 0,1,•••, N - 1 , m =1,•••, M with
Q.. quantization levels in m -th signal eomponent:

(7.1.1)

where Ö (.) is Kronecker delta, k is index of signal sampIe co-ordinates and

m is index of signal components or signal sampIes in the groups. In
applications, it is very frequently sufficient to compute and to use histograms
without normalization by the number of signal sampies. We will denote
them ii(iiM):

(7.1.2)

Sometimes histograms should be eomputed over signal sampIes weighted

with some weights {w k }. We will refer to these histograms as weighted
histograms ii,.{ijM):

(7.1.3)

In image processing, when histogram is computed for an entire image

frame or over a set of images it is referred to as global histogram. Many
image proeessing algorithms are based on histograms computed for every
pixel over a set of surrounding sampIes called pixel spatial neighborhood
(see eh. 12). Such histograms are referred to as loeal histograms.
If local histograms should be computed for every pixel in the process of
image scanning, computations ean be earried out recursively. Let local
histograms be eomputed in a rectangular window of (2N! + 1)x (2N 2 + 1)
 7. STATISTICAL COMPUTATION METHODS AND ALGORITHMS 277

pixels in the process of regular scanning image array {ai~)} row-wise and
then column-wise on the rectangular sampling grid. For each I-th row,
compute first histograms ~(k )(qm) of window columns:

(7.1.5)

Then local histograms ~,1(q.. ) can be recursively computed as

(7.1.6)

The interpretation ofthis formula is self-evident: local histogram ~AqM)

in the window centered at pixel (k,/) may be obtained from the histogram
~,1-1(qM) of the in the window in the previous position (k,I-1), if it is
added to the difference of column histograms ~~k22 (qm) and ~~22-1 (qm) of
window columns that enter window and leave it in the process of window
shift from (k,l-l)-th to (k,/) position. Column histograms Ii}k)(ii.. ) may
be, in their turn, also computed recursively:

(7.1.7)

from column histograms on the previous row. Thanks to such a recursive

computation, computational complexity of computing image local
histograms may not depend on the sliding window size provided memory
buffers for storing column-wise window histograms are available. Otherwise
it is proportional to vertical size of the sliding window when only row-wise
recursive computation is implemented.
Useful statistical characteristics of signals are histogram moments. We
will describe them for single component signals. n - th histogram moment
ii(n) of signal sequence {a k } is defined as:
278 Chapter 7

(7.1.8)

It follows from this definition and from the definition of the histogram
(Eq.7 .1.1) that histogram moments can be directly computed from signal
sampies as:

-(lI) _ 1 ~( \11 (7.1.9)

a --L..J akJ .
N k=O

Signal first moment is its mean value. Signal second moment is called
signal variance. Square root of the variance of signals centered around their
mean value

cr 1 1-- ~ii0(2) -
-
~ii(2) - 'a-(I)'\2
~
. a - a -ii(l) .
)'0- (7.1.10)

is called signal standard deviation.

Like the distribution histograms, the moments may be employed as
global characteristics of the entire image or of an ensemble of images or as
local characteristics of image fragments. Local moments of image array
{ak,l} in a window of (2N!W) + 1X2N~W) + 1) pixel sliding over the image
array row-wise and then column-wise:

(7.1.11)

are local means of signal {(a k .,)"}. Therefore recursive algorithm for
computing signal local mean described in Sect. 5.1.1 can be efficiently used
for computing signallocal moments.
Digital signal statistical characteristic alternative to histograms and
moments are order statistics. Signal order statistics are defined throußh a
notion of the variational row. Variational row VR{lId of a signal tak},
k =0,1,.•., N -1 is obtained by ordering signal sampies according to their
values in a monotonically increasing sequence:

(7.1.12)
 7. STATISTICAL COMPUTATION METHODS AND ALGORITHMS 279

If signal sampIes happen to have the same quantized value they are
positioned in the variational row one after another in an arbitrary order, for
instance, in the order they were accessed in the process of the variational
row formation. Note that coordinate indices {k} of signal sampIes are
ignored in the variational row.
Element a(R) that occupies R-th place in the variational row is called
signal R -th order statistics. Index R (R = 1,2,•••, N ) of the order statistics
is called its rank. The rank shows how many signal sampIes have value
lower or equal to that of the element. First order statistics is signal minimum:

(7.1.13)

Last order statistics in the variational row is signal maximum:

(7.1.14)

If the number of signal sampIes N is odd, (N + 1)/2 -th signal order

statistics that is located exactly in the middle of the variational row is called
signal median

a«N+I)/Z) = MEDN {at}. (7.1.15)

If N is even number, median may be defined as

(a(NIZ-I) +a(NIZ-I»)1 2 = MEDN{ak }. (7.1.16)

Rank of a sampIe with quantized value q. can be found from signal

histogram 1i(q) as

(7.1.17)

Signal median may be regarded as an alternative to the signal first

moment, its mean value. Yet another alternative to signal mean is signal
alpha-trimmed mean ([1]):

_ _ 1 N~--<X (r)
aa. - - - k.Ja , (7.1.18)
N -Cl r=o.
280 Chapter 7

the arithmetic mean of the elements of the variational row which are
minimum (l elements apart from its ends.
Signal second moment, or its variance, is a parameter that characterizes
spread of the signal histogram. An alternative characteristics of signal values
spread based on order statistics is signal quasi-spread ([1]):

QSPR .. (a)=a
(-N+I) (N+I)
1+.. -a - 1- ..
(7.1.18)

with n as a parameter.
In image processing, order statistics and ranks can be measured both
globally over entire image or set of images and over a sliding window. In the
latter case they are called local. Local histograms, ranks and order statistics
are the base ofnonlinear filters discussed in Ch. 12.
Local ranks are usually computed for the window central pixel. Because
local ranks can be obtained from local histogram (see Eq. 7.1.17), they may
be computed in a recursive way. They mayaIso be calculated recursively not
through the local histogram, but in a direct manner as weIl. For this purpose,
it suffices to calculate the difference between the number of window
elements appearing at a given scanning step with a value smaller than that of
the window central pixel, and the number of such elements disappearing
from the fragment, and add this difference to the rank obtained at the
previous step.

7.1.2 Measuring signal correlation functions and spectra

Auto-correlation functions, cross-correlation functions and power spectra

as signal characteristics appear in solving problems of image restoration and
object localization with the use of linear filtering (see Chs. 8, 10, 11). Being
statistical characteristics, they assume signal treatment as random processes
when signals and images subjected to processing are regarded as
representative or realizations of a certain signal/image ensemble or data
base.
Let {a", (x)} is an ensemble of signals specified on interval X with
ro E n as an ensemble n parameter. The ensemble may be represented by
a set of signals in a data base or it may be defined by a certain mathematical
statistical model. The autocorrelation function of the signal ensemble is
defined as:

(7.1.19)
 7. STATISTICAL COMPUTATION METHODSANDALGORITHMS 281

where AV", 0 is an operator of averaging over the ensemble. Cross-

correlation lunction of two signals a", (x) and b", (x) is defined similarly
as:

(7.1.20)

Fourier transfonn of signal autocorrelation fimction

00

Pa (I) = fRa (I; )exp(i21tft)d1; . (7.1.21 )

is called signal ensemble power spectrum. Altematively, signal power

spectrum may be found directly from the signal through its Fourier
Transfonn as

(7.1.22)

In this case signal correlation function is computed as Fourier transform of

its power spectrum:
00

Ra (I; ) = f Pa (I )exp(- i21tft ~ . (7.1.23)

Discrete representation of correlation functions and spectra through signal

sampIes may be obtained similarly to discrete representations of signals
discussed in eh. 4. For signals a", and b", specified by their sampIes {a", ,k}
and {b",,Je }, k = 0,1, ..., N -1, autocorrelation function, cross-correlation
fimction and power spectrum are computed, respectively, as:

(7.1.24)

(7.1.25)
282 Chapter 7

P" (r) = 1
r;;;
vN
L RII (n )exp(nrJ
N-l

n=O
i2n - ,
N
(7.1.26)

where ro ,as before, is a signal ensemble parameter, or, altematively, as

(7.1.27)

and

RI (n) =
1
r;;;
vN
L p" (r )exp(nr)
N-l

n=O
- i2n -
N
(7.1.28)

Obviously, in these computations boundary effects associated with the

finite length of signals should be treated in ways similar to those discussed in
Sect. 5.1.4 for digital filtering. The definitions of signal power spectrum
given by Eqs. 7.1.26 and 7.1.27 assume signal periodical replication. If
signal correlation function is computed through the DFT of its power
spectrum it also will assume signal periodical replication.
Boundary effects in evaluation of signal correlation functions and spectra
may be rather severe. It is especially true in image processing. One of a
popular solutions of this problem is evaluation signal power spectrum and
correlation functions using signal and spectrum windowing as follows:

(7.1.29)

and

RII(n)= 1
r;;;
(nr) ,
LW}P)p,,(r)exp -i2n -
N-l
(7.1.30)
vN n=O N

where W}II) and W}p) are window functions that are equal to one in the
middle and the largest part of the interval 0 + N and decay more or less
rapidly to its ends. They are discrete analogs of optical apodization masks.
Among the different possible window functions the most known are
Hamming window:

W =0.54+ 0.46cos[n (n-N 12)/N]

n (7.1.31)
 7. STATISTICAL COMPUTATION METHODS AND ALGORITHMS 283

and Kaiser window:

W = 10(a~I-[(n-N/2)/N]2)
(7.1.32)
n lo(a)

0
where 1 0 is the modified zero order Bessel function of the first kind and
a is a parameter.
Hamming and Kaiser window functions do not decay to zero at ends of
interval 0 + N . They are applicable as spectral windows. As signal window
functions, they are applicable for signals with zero mean value. For images,
window functions that decay to zero are more appropriate. Two possible
examples are functions:

0.5 + 0.5 cos[n (K - n)/ K 1 n = 0,1, ••, K

Wn = 1,
{ n = K + 1,.•., N - K - 2
0.5+ 0.5cos[n(n- N + 1 + K)/ Kl n = N - K -1,...,N-l
(7.1.33)

and

Wn =;atan
2 - 2cr /2)]
[ ß exp(n-N 2 (7.1.34)

with K ,cr ß and as spread and speed of decay parameters.

For 2-D signals and images, window functions may be buHt as separable
to product of 1-0 functions. Better results with lower boundary effect
artifacts can be achieved with inseparable circularly symmetrical window
functions.
Yet another option for evaluating signal power spectra with reduced
boundary effects is the use of the OCT as areplacement for the OFT in the
same way and on the same base as as it was recommended in Sect. 5.2.2 for
signal convolution.
In image processing it is very frequently necessary to evaluate signal
correlation functions and spectra with sub-pixel accuracy, that is, not only in
the sampling points but in intermediate points as weIl. This can be achieved
by methods of signal discrete interpolation discussed in Ch. 9.
284 Chapter 7

7.1.3 Measuring parameters of random interferences in sensor and

imaging systems

One of the tasks in processing images, holograms and interferograms is

suppressing random interferences introduced by the imaging, holographie
or interferometric sensorie systems. For this, one usually needs to know
parameters of the interferences. Sometimes the required data are provided by
manufacturers of the systems. In practice, however, such data are often
lacking, and parameters of signal interferences need to be derived direct1y
from signals to be processed.
At first sight, this problem may seem to be internally contradietory,
because in order to evaluate parameters of noise, one must separate the
noise from the signal, which may be done only by knowing the parameters
of this noise. A way out of this "vicious circle" is not in separating the signal
from noise in order to obtain statistieal characteristics of the latter, but
rather, in separating signal and noise characteristies based on measurements
of the corresponding characteristics of the noisy signal.
Luckily, in most practieal cases random interferences, statistieally
speaking, are very simple objects; that is, they are described by a very small
number of parameters. For instance, additive signal independent white
Gaussian noise is fully specified by only two parameters: by its mean and
variance. Because of this, noise parameter estimation problem may often be
solved by rather simple means. Basic principle is choosing those signal
characteristics in which signal distortion by noise manifests itself in an
abnormal behavior that can be detected in the easiest way.
All detection methods are based on an apriori assumption for certain
smoothness and regularity of the characteristics of undistorted signal that are
violated when signal contains noise. Two practieal detection methods are the
predietion method, and the voting method for detecting anomalies.
The predietion method requires that for every element of the analyzed
sequence, the difference between its actual value and the value predicted by
previously investigated elements be found. If this difference exceeds so me
given threshold, adecision is made that an abnormal burst has been
encountered. The predietion depth, the method of obtaining apredieted
value and the threshold value are method parameters that are to be specified
apriori.
The simplest prediction technique employs the value of the previous
sequence element as the predieted value of the analyzed sequence element.
Prediction mayaIso be carried out by means of summing several previous
values taken with a certain weight on each. In this case the optimal values of
the weight coefficients are found from the conditions of preserving the first,
the second and higher differences of the analyzed sequence, or from
some auto-regression models built empirically from signal data base.
 7. STATISTICAL COMPUTATION METHODS AND ALGORITHMS 285

The voting method implements ideas of robust estimation ([1]). The

essenee of the method is in eonsidering eaeh element of the analyzed
sequenee together with eertain number of its neighbors. This set of elements
is ordered in a variational row and eheeked whether or not the value of the
given element belongs to a eertain predefined number of extreme (maximal
or minimal) value of the variational row. If the answer is yes, then a
decision on the presenee of an abnormally high (or, alternatively,
abnormally low) value in the given element is made. The voting method is
therefore based on the assumption that deviations, if any, from loeal
monotony of a "normal" eharaeteristie under study are insignifieant. The
number of neighbors and the deteetion threshold are to be speeified apriori
from the study ofa "normal" behavior of the se lee ted eharaeteristie ofthe
undistorted signal.
We will illustrate these approaehes using examples of parameter
estimation of additive broad band and narrow-band noise, impulse
noise, and quantization noise.

7.1.3.1 Parameter estimation of additive signal-independent broad

band noise in images.
Additive statistieally signal-independent noise with normal distribution
is fully specified by its standard deviation and auto eorrelation funetion. If
the noise is uneorrelated or weakly eorrelated (broad band), as is often the
case, then its varianee and correlation function may be found through the use
of the following simple algorithm based on measuring abnormalities in the
eovarianee function of the observed image.
Let it is known that observed signal {aro .,/<} is eontaminated by adding to
a "pure" signal {aro.,k} of a signal ensemble 0a zero mean signal
independent noise {nro .,/<} ffi n E On of noise ensemble with eorrelation
funetion Rn (k) :

(7.1.35)

Correlation funetion of signal {aro .,/<} is:

AVrod ( -N1 t; a
N-l
ro.,k
a'ro.,k+r J+ AVro. (-N1 t; n
N-I
ro.,k
n'ro.,k+r
J+
 286 Chapter 7

One can see from Eq. 7.1.36 that, owing to noise additivity and statistical
independence of the signal, the correlation function of the observed signal
Ra (r) is a sum of the correlation function the noise-Iess signal RAr), the
correlation function Rn (r) of the noise and of the realization of some
"random" process e n' The latter describes the error in measuring the noise
correlation function that are in practice carried out over its finite-size
realization rather then over the entire ensemble. The variance of the random
process e n is known to be inversely proportional to the volume of the
realization used for the measurement. Normally, in image processing
hundreds of thousands of pixels may be involved in the measurement.
Therefore, the variance of random error e n in (7.1.36) is usually small
enough and Rn{r) may be, with a good accuracy, estimated as

(7.1.37)

Consider, first, the uncorrelated noise case when,

(7.1.38)

where (J; is the noise variance. In this case, the correlation function of the
observed image deviates from the correlation function of the noise-Iess
image in the coordinate origin only, and the difference is equal to the noise
variance:

(7.1.39)

For all other values r the magnitude of R ö (r) may serve as an estimate of
Ra (r). Assuming that Ra (r) is a monotonie function in the vicinity of
r = 0, one can use Rö (r) = Ra (r) for r > 0 to extrapolate its value Ra (o)
for r = 0 and then use the extrapolated value to compute noise variance
from Eq. 7.1.39.
A similar approach may be used to estimate the variance of a weakly
correlated (broad band) noise, i.e. a noise whose correlation function Rn (r)
 7. STATISTICAL COMPUTATION METHODS AND ALGORITHMS 287

is nonzero only in a small vicinity around the coordinate origin. In this

vicinity, the values of the noise-less image correlation function may be quite
satisfactorily obtained by extrapolating R ii (r) from points where Rn (r) is
known to be zero. Fig. 7-1 illustrates this idea on an example of measuring
noise in interferograms.

c)

Figure 7-1. Measurement noise level in an interferogram: a) - noise-Iess interferogram; b) - its

1-0 correlation function; c) - noisy interferogram; noise variance 10000; d) - 1-0
autocorrelation function ofthe noisy interferogram. One can c1early see an anomalous peak in
the correlation function ofthe noisy interferogram.

7.1.3.2 Estimating intensities and frequencies ofspectral components of

periodic and similar narrow-band noise.
Additive periodic (moire) interferences are very characteristic for
electronic imaging systems and image sensors. A feature typical for this type
of interference is that their Fourier spectrum contains just a few components.
On the other hand, the spatial spectrum of noise-free images is generally a
288 Chapter 7

more or less smooth function. Therefore, the presence of a narrow-band

noise in images manifests itself in the form of abnormally large and
localized peaks in image power spectra and it is image power spectrum that
has to be analyzed for the noise diagnostics.
In contrast to the case considered above in which the broad band noise
produces anomalous peaks in the correlation function only in the vicinity of
the coordinate origin, the locations in spectral domain of peaks of noise
spectrum are usually unknown and have to be determined. This may be
achieved with the help of the prediction or voting techniques described
earlier.
We will explain the diagnostics of the periodical noise in images using as
an example an image shown in Fig. 7.2 a) .
.- Averagld power 'paotr1Jm
2!Dl ••. ,- - --. - - - -, --- .----,----, -"', • •• ,-- -,-

-r-· -+---~- - --i. _--~ ---+- --~_. --f-

, I , I I I j I •

nl) .•• •• •~
1!m .•. ~ •••• : .• • . :- .•• ~ •• --: .••. : ... . :- . . • ~ .... : .
• •

1(D] ---~ . ---~---+---~-_.-:

I , I

.. _-:-_ ..~ ._- ~ _. _.:.

, I • ,

-.,. --T "'r---: _.a -; ----: --ar ---I-

• • , • I • I • ,
• • • • , • I I ,
-_.~.

3l 4050 61170 EIJ90

·
I
. . ..
• , I I
,..
I I •

---;----r---T---r---r---r-·-T--1----!-
I • • , . I I •

3D
2IXJ --.!----~.---:_---~-_.-!----~----:--
• • • , I • I
-- -. ,.
• • • • • • I

'00 --- ~ - - --~- - --~ - - - ~ - - - - t -. - - ~ ----:-.

• I • • • • •

OL-~
1 0~~=-~3l~7.
40~50
=-~611~~70~OO~~90~
Frequency index
~ ~
Figure 7-2. Diagnostics ofperiodical interferences: a)· noisy image; b) averaged column
power spectrum ( top) and the resuIt of estimation of spectrum of the interference (bottom )

The image is distorted by a periodical interference. The top graph of

Fig.7-2, b) shows image I-D power spectrum obtained by computing
squared absolute values of DFT spectra of image columns and summing up
spectra of all columns. One can easily see a peak in image high frequencies
that can be attributed to the interference. In order to detect this peak and
mark spectral components affected by the interference, one can analyze this
spectrum beginning from low frequencies employing the prediction or voting
techniques. By virtue of noise additivity, the intensity of each distorted
spectral component Pa (r) is equal to the sum of the intensity Pn(r) of the
corresponding spectral component of noise and of that Pä (r) of noise-free
 7. STATlSTICAL COMPUTATION METHODS AND ALGORITHMS 289

signal and that. The latter may be estimated by interpolation between those
image spectral components that are found not affected by the interference
assuming smoothness of noise-free image power spectrum. Power spectrum
ofthe interference may then be found as:

(7.1.40)

This is illustrated in Fig. 7-2, b, bottom graph.

The same approach with analysis of image power spectra may be
employed for diagnostics of additive broad band noise in interferograms
with a spatial carrier, illustrated by an example shown in Fig. 7-3, a). In
spectra of noisy interferograms, noiseless interferograms will manifest
themselves in concentrated peaks in spectral domain (see Fig. 7-3, b)) that
may be easily detected and marked. Then noise variance may be
straightforwardly evaluated over the area in spectral domain outside the
marked signal peaks.

Figure 7-3. Noisy interferogram with a spatial carrier (Ieft) and its power spectrum displayed
as an image (right).Vertical and horizontal directions in this image correspond to vertical and
horizontal frequencies, respectively while brightness represents spectrum intensity.

7.1.3.3 Parameter estimation ofimpulse noise, quantization noise.

As it was indicated in Sect. 2.8.1, the major statistical characteristic of
impulse noise is the probability of signal sampie distortion. It may be
estimated through the relative number of the picture elements affected by
the noise bursts. These noise bursts may be found with the help of the
prediction and voting techniques applied to the analysis of the sequence of
noisy image sampie values in a small neighborhood of every pixel. Thus, the
290 Chapter 7

procedure for estimating the impulse noise parameters may be combined

with that of filtering. Impulse noise diagnostics and filtering are treated in
more details in Chapter 8.
Quantization noise is determined by the number of signal quantization
levels. To find the number of these levels, it suffices to calculate the
distribution histogram of the signal values and find the number of such
values that corresponds to non-zero sampIes of the histogram. In a similar
way, employing the prediction or voting techniques to find the dips in the
histogram and measuring the depth of such dips, we may estimate the
probability of failure of any quantization level which is characteristic of
some digital imaging systems. Fig. 7-4 shows, as an illustration, an
interferogram quantized to 256 gray levels and its histogram. One can easily
see in gaps between bars in the histogram that indicate that in reality the
interferogram has only about half of 256 levels allocated in the quantization.

.........................................
• I I I t

.. .. .. .. . . . . .
• I
... . . .. .. .. .. .. .. ..
• •
..
•

.. .. .. L ....... L ...................... J

......................................
• I • I

• • I I

.....
Figure 7-4. An interferogram quantized to 256 gray levels and its histogram
- ,'''
 7. STATISTICAL COMPUTATION METHODSANDALGORITHMS 291

7.2 DIGITAL STATISTICAL MODELS AND MONTE

CARLO METHODS

7.2.1 Principles of generating pseudo-random numbers. Generating

independent uniformly distributed pseudo-random numbers

Digital modeling is widely used for modeling optical signals and

systems. Modeling may be either deterministic or statistical. In statistical
modeling, the final result is obtained through averaging of the characteristics
and data acquired in a set of experiments. In deterministic modeling, the
result of each model experiment is treated individually; no averaging over a
set of experiments is assumed. Therefore, unlike deterministic models,
statistical digital models inc1ude generators of signals that are regarded
random.
In statistical modeling, "random" images and objects are generated in the
form of arrays of so called "pseudo-random" numbers with pre-defined
statistical properties. This means that in this type of modeling it is not the set
of specific values of the array elements that are specified but, rather, only
certain parameters or characteristics averaged over all realizations of this
sequence. The choice of parameters depends upon the nature of the modeling
problem we are dealing with. Even though in digital modeling any concrete
sequence or realization may be reproduced with absolute accuracy, the effect
of any specific set of values of the sequence elements on the modeling result
is not considered. Statistical simulation with the use of pseudo-random
numbers is referred to as Monte-Carlo simulation.
As a rule, description and specification of images and wave fields as
random objects are carried out with the help of such commonly known
statistical characteristics as one- and multi-dimensional probability
distribution functions, correlation functions and power spectra. The question
we consider next is how one can generate realizations of pseudo random
sequences with pre-specified statistical characteristics. A general method
rests upon an assumption that pseudo-random sequences with arbitrary
statistical characteristics may be generated from elementary sequence of
statistically independent pseudo-random numbers by means of applying to it
appropriate linear or non-linear transformations.
The most widely used algorithm for generating statistically independent
pseudo-random numbers is a recursive one: each new number ; k in the
sequence is generated from the number ; k-l obtained on the previous
recursion step as following:

(7.2.1)
292 Chapter 7

where Cl' C 2 and C 3 are constant parameters of the algorithm [2]. For
generating 2-D pseudo-random arrays of numbers, the arrays are
successively filled in row-wise / column-wise.
The initial number, or seed, generally has a little effect upon the quality of
the sequence thus obtained. However the algorithm is very sensitive to the
parameter especially to parameter C 3 that plays a decisive role in
determining the period of repeatability of the generated numbers. Provided
appropriately selected parameters, the algorithm generates pseudo-random
numbers that do not exhibit noticeable correlations and have approximately
uniform distribution. We will refer to this algorithm as "primary pseudo-
random number generator". It is usually used to generate pseudo-random
numbers with any special statistical properties.

7.2.2 Generating pseudo-random numbers with special statistical

properties

In applications, pseudo-random numbers with different distribution

functions or/and different correlation functions are needed. They are usually
generated by means of appropriate transformations of the numbers obtained
with primary random number generator.

7.2.2.1 Generating pseudo-random numbers with special distribution

function
Pseudo-random numbers {I;} with known distribution density
p(~) can be converted into numbers {!,} with distribution density
p(,,) by element-wise nonlinear transformation ,,= F(~) that is
defined by the differential equation:

p(1'\ }dF = p(;}d; . (7.2.2)

Flow diagram ofthe algorithm is shown in Fig. 7-5.

If primary random number generator produces numbers with uniform
distribution in the range ~ min + ~ max output pseudo-random numbers
will have distribution

(7.2.3)
 7. STATISTICAL COMPUTATION METHODS AND ALGORITHMS 293

Output pseudo-
Primary pseudo- . , Poiot-~e randorn nurn bers
randorn nurnber p
Doo)ineatity ...
r" .. , -"-
generator o:t .~Y· ~ .

Figure 7-5. Flow diagram of an algorithm for generating pseudo-random numbers with a
given distribution function

For example, exponential transformation

F(~ ) = exp(~) (7.2.4)

enables generating, from uniformly distributed pseudo-random number,

pseudo-random number distributed in accord with the hyperbolic law:

(7.2.5)

with therange (TImiD =eXP(~mIJ,Tlm.. =exp(~m.. ))·

An important sub-class of element-wise transformations, one that
accomplishes binarization of values by thresholding

F(~)= {I,0, ~ ~ T~r;

otherwlse
(7.2.6)

is useful for generation binary pseudo-random numbers. For standard

primary pseudo-random generator with uniform distribution in the range
(0 + 1), parameter Thr of the algorithm is determined by the required
probability P(l) of ones: Thr = P(l) .
If transformation function F(~) remains the same with all sampIes of the
input sequence (array) of pseudo-random numbers, than the output sequence
(array) is stationary, or homogeneous. Inhomogeneous arrays of pseudo-
random numbers may be obtained by arranging to alter the transformation
function in a certain controlled way.
 294 Chapter 7

7.2.2.2 Generating pseudo-random numbers with normal distribution

and specified correlation functions and power spectra
Pseudo-random numbers with normal distribution function and special
correlation function and spectra are of a special importance in statistical
simulation of noise phenomena and in statistical modeling images and wave
fields. The major method for generating such numbers is linear filtering of
arrays of numbers generated by the primary random number generator
according to flow diagram shown in Fig. 7-6.

Output pseudo-
Primary pseudo- rando m numbers
random number ""-
Linear filter ""-

generator

Figure 7-6. Flow diagram of an algorithm for generating pseudo-random numbers with a
given correlation function

Correlation function ofpseudo-random numbers {!,k}

(7.2.7)
n

obtained by filtering non-correlated pseudo-random numbers {l;k} with

correlation function R~ = 0':8 (n) using digital filter with PSF {h n} is may
be found as:

~ (n) = A ~ (11 kll k+n ) = A ~ ( ~ ~ h",h/~k_"'~k+n_/ ) =

I I hmh/A ~ (~k-"'~k-/)= I I
m /
hmh/0':8 (m + n -I) = 0': I m
hmhm+n ,

(7.2.8)
where 8 (.) is the Kronecker delta-function. Linear filtering results also in
"normalization" of the distribution function of numbers {llk}' By virtue of
the central limit theorem of the probability theory, the higher is the number
of terms involved in weighted summation of initial pseudo-random numbers
for computing each of the numbers {ll k} according to Eq. 7.2.7, the c10ser is
their distribution function to the Gaussian (normal) one.
As it was described in Sect. 5.2.1, convolution is frequently more
efficiently implemented as cyc1ic convolution in the OFT domain. This
 7. STATISTICAL COMPUTATION METHODS AND ALGORITHMS 295

suggests an alternative algorithm for generating correlated pseudo-random

numbers described by flow diagram in Fig. 7-7. This algorithm enables
generating groups of correlated pseudo-random numbers with specified
power spectrum and distribution function that, for large group size perfectly
approximate the Gaussian (normal) distribution function. The computational
complexity of the algorithm is O(log 1 N) operations per one pseudo-random
number of the group of N numbers,.

Primary Point wise muItipIicatioD by

generator of ~ ~ ~
weigbt coefficients defined F'FT
psendo-random
number by thc required power
spedrum

Figure 7-7. Flow diagram ofthe algorithm for generation pseudo-random numbers with
normal distribution and given correlation function

. ,. . .. ' ., .
:

Figure 7-8. Empirical distribution function of 65000 pseudo-random numbers with Gaussian
distribution generated by the FFT method

In order to explain the algorithm, let ~}re)} and ~~Im)} be two arrays of
N non-correlated pseudo-random numbers. Form out of them an array of
complex numbers ~~re) + i~1Im)}, multiply these numbers by some weight
coefficients {wk } and compute DFT ofthe result:

(7.2.9)
296 Chapter 7

By virtue of the central limit theorem of the probability theory,

distribution function of pseudo-random numbers ~~re)} and ~ym)} tends,
with the increase of N, to the Gaussian (normal) one for practicallr
whatever distribution function of the initial pseudo-random numbers ~;re) J
and ~ym)}. In practice of Monte Carlo simulation in image processing and
holography, the dimension N of arrays amounts to tens and hundreds of
thousands. Therefore the quality of approximation of normal distribution
function is usually extremely high. Fig. 7-8 illustrates this feature of the
method. Graph on the figure shows distribution function of 65000 pseudo-
random numbers generated by the described method. The graph is plotted
coordinates in which true Gaussian distribution corresponds to the diagonal
straight line. One can see from the graph that the empirical distribution
slightly deviates from the diagonal only for probabilities of the order of
10-4 •
For zero mean and mutually non-correlated numbers ~~re)} and ~~im)}
with the same variance (j t = AV~ (S~re»)= AV~ (S~im»), correlation functions of
numbers ~~re)} and ~~im)} and their mutual correlation function are
determined by weight coefficients {w k }:

(7.2.10)

(7.2.11)

where (j t = AV~ (sire »)= AV~ (s2mg »).

Select weight coefficients {W k } such that {W k = W N - k }. Then obtain:

(7.2.12)
 7. STATISTICAL COMPUTATION METHODS AND ALGORITHMS 297

and

(7.2.13)

Therefore, squared weight coefficients {w k } define power spectrum of

numbers ~!re)} and ~!im)} and these sets of numbers are mutually non-
correlated.
As it was already mentioned, the computational complexity of the FFT
method is O(log 2 N) operations per one of N numbers assuming using a
FFT for computing the DFT. We will refer to this method as FFT method
for generating correlated numbers with Gaussian distribution. An
important advantage of the FFT method, in addition to its capability to very
accurately imitate Gaussian distribution with low computational complexity,
is the fact that it is not very sensitive to the distribution function of the
primary pseudo-random numbers and to their possible correlations. These
correlations, if they exist, will not propagate to output and may cause only
certain inhomogeneity of the resulted arrays of correlated Gaussian numbers.

7.2.3 Generating pseudo-random images

One of the applications of the above-described methods for generating

pseudo-random numbers with specified statistical characteristics is
generating pseudo-random images that imitate natural texture images. a
constructive approach to solving this problem is an algorithmic one
illustrated in Fig. 7-9 ([3]). According to this approach, any texture image
may be created by means of an appropriately designed transformation of a
primary generating texture produced by the primary pseudo-random number
generator. We will illustrate this approach using several examples.

Texture ima ge
Primary generator
of pseudo-random
Transformation ....
system
Dumbers

Figure 7-9. An algorithmic approach to synthesis of texture images

Texture images shown in Fig. 7-10 were generated by linear filtering ofa
primary array of uniformly distributed uncorrelated pseudo-random numbers
by the above-described FFT method. Image (a) was produced by linear
filtering with circular frequency response shown in the black box at the side
 298 Chapter 7

c) d)

Figure 7-10. Correlated texture images generated by the FFT method.

c)

Figure 7-11. Imitating natural textures: a), b) - images ofnatural textures; c), d) - generated
images
 7. STATlSTlCAL COMPUTATION METHODS AND ALGORITHMS 299

of the image. The radius of the circle is equal to 0.1 of the highest spatial
frequency. Image (b) was produced using linear filter with frequency
response in a form of a ring shown in the black box. External radius of the
ring is 0.1 and internal radius is 0.075 of the highest spatial frequency of the
image.
Images (c) and (d) exemplify fractal images with power spectrum that
decays with spatial frequency index , as ,-1
and ,-I.S respectively. Such
images can be used to imitate clouds of different density.
Images shown in Fig. 7-11, (c) and (d) show textures that imitate natural
texture image of textile (a) and wood (b), correspondingly. They were
obtained by assigning power spectra of natural texture images to images
generated from primary independent random numbers.

a) b)

Figure 7-12. Inhomogeneous and composite synthetic texture images

Image shown in Fig. 7-12, (a) represents an example of a spatially

inhomogeneous texture generated by the method of recursive filtering in
sliding window described in Sect. 5.4. In each position of the window,
central pixel of the window is generated by inverse FFT after assigning to
primary pseudo-random numbers in the window an anisotropie power
spectrum. Orientation of the anisotropy was controlled by another texture
image ofthe type shown in Fig. 7-10, (a).
Image in Fig. 7-12, (b) is a complex texture image composed of two
textures. The background texture is a synthetic texture similar to that shown
in Fig. 7-9, (c) that imitates textile. The foreground image was obtained by
convolution, with a template in a form of a coin, of a binary texture image
with probability of ones 4.10-4 •
300 Chapter 7

7.2.4 Generating correlated phase masks. Kinoform and

programmed diffuser holograms

Correlated phase masks are frequently needed in digital holography. Two

typical examples are kinoform and programmed diffuser holograms.
Kinoform is a phase only computer generated hologram ([4]). In synthesis
of kinoform, a pseudo-random array of numbers in the range (-1t,1t) is
generated and recorded on a phase media as a hologram phase profile. The
array should be selected in such a way as to reconstruct from the kinoform,
in Fourier plane of an optical reconstruction setup, a required image, that is,
a certain given distribution oflight energy.
"Programmed diffuser" method for synthesis of computer generated
display holograms simulates diffuse light scattering from an object surface
by assigning to the object wave front a correlated pseudo-random phase
component with power spectrum that is determined by a given scattering
directivity pattern of the object diffuse surface ([5]). Diffuse surface
directivity pattern is the distribution, in the far (Fourier) zone of diffraction,
of intensity of radiation scattered by the surface uniformly illuminated by a
plane wave.
Let {s kJ be an array of numbers in the range (-1t, 1t) representing the
required phase mask and {P(r, s)} is a discrete representation of the required
distribution of light intensity in the Fourier domain. In both above cases, as
weH as in general in the synthesis of corre1ated phase masks, it is required
that

P(r, s) =C 'IDFT({exp(i21tS k,/ )}f ' (7.2.14)

where C is an appropriate normalizing constant. Given {P(r, s )}, {s k,/ }

should be found as a solution of Eq. 7.2.14. In general, the exact solution of
the equation may not exist. However, one can always replace it by a solution
~k'/}' taken from a certain dass of phase distributions, that minimizes an

appropriate measure D(.,.) of deviation of C ·IDFT~XPV21t§k,/)W from

{P(r,s)}:

~k'/}= ar~kjin[ D( {P(r,s )},C ·IDFT[~XP(i21t§k./ )}W)]· (7.2.15)

Pseudo-random number generator may serve as a source for realizations of

the phase distributions and a solution ofEq. 7.2.15 may be searched through
an iterative procedure ([6]) according to a flow diagram shown in Fig. 7-13.
One can also inc1ude in this process, when it is necessary,
 7. STATISTICAL COMPUTATION METHODS AND ALGORITHMS 301

Computing rhase
I Primary array of pseudo-random numbers

....
~ !~) }
I
~(I)
k,/ ,
H.
- Quantization
- Computing A(,- I )(r, s)exp(i ~ (r, s)}) = DFT(exp~ ~1~/1) }D

- Computing d(') = D[IA(I-I)(r,s f;p(r,s)]

IDFT
~~
I - Comparing d(') and d('- I)

Replacing A('-I)(r,s)
by
t ~k,/ }
[P{r,s)1'2;0 ~ P(r,s) ~ 1 ~ Continue iterations?
No
Yes

Figure 7-13. Flow diagram of an iterative algorithm for generating pseudo-random phase
mask with a specified power spectrum.

a) b)

~
0
0.3

~c 0 .25 -
0
0 .2
~
~
0 . 15
A 0. '
üi
0 .05

0
5 '0 '5 20 25
h:Ol'aUon ..
c) d)

Figure 7-14. Examples of images reconstructed from kinoforms: a) original image used for
the synthesis of kinoforms; b) and c) - reconstructed images for kinoforms with 255 and 5
phase quantization levels, respectively; d) plots of standard deviation of the reconstruction
error as a function of the number of iterations for no quantization, 255, 15 and 5 phase
quantization levels
302 Chapter 7

quantization of the phase distribution to a certain specified number of

quantization levels.
Flow diagram of Fig. 7-13 assumes that iterations are carried out over
one realization of the array of primary pseudo-random numbers. In principle,
one can repeat the iterations for different realizations and then select of all
obtained phase masks the one that provides the least deviation from the
required distribution {P(r, s)} .
Illustrative examples of images reconstructed, in computer simulation,
from kinoforms synthesized by the described method with 255 and 5 phase
quantization levels are shown in Fig. 7-14, b) and c) along with original
image (Fig. 7-14, a) used as an amplitude mask [P{r,s)]112. In this
simulation, mean squared error criterion was used as a measure of deviation
of the reconstructed image from the original one. On the graph of Fig. 7-13,
d), one can also evaluate quantitatively the reconstruction accuracy and the
speed of convergence of iterations. The graphs show that iterations converge
quite rapidly, especially for coarser quantization and that the reconstruction
accuracy for 255 phase quantization levels can be practically the same as
that for kinoform without phase quantization. Coarse quantization of the
kinoform phase worsens the reconstruction quality substantially.
In the programmed diffuser method, correlated random numbers are
needed to simulate, in the synthesis of holograms, a property of object
surfaces to non-uniformly scatter irradiation and in this way to provide a
visual clue about the object shape in images reconstructed from holograms.
The method assumes the observation scheme The method assumes the object
observation scheme shown in Fig. 7-15 and defines the object by two array
of numbers. The first array A{x, y)describes object's reflectivity as a
function of coordinates (x, y) in a plane tangent to the object surface and

Figure 7-15. Scheme of observation of diffusely scattering objects

7. STATISTICAL COMPUTATION METHODS AND ALGORITHMS 303

perpendicular to the direction to the observation (hologram) plane (I;, rJ) ..

The second array describes object' s surface depth z(x, y) with respect to this
plane. In addition, an array of correlated pseudo-random numbers
PrDif/(x,y) is generated, as it was described above, with power spectrum
defined by the required directivity pattern of the object surface. Then the
Fourier hologram is computed for the object wave front

A(x, y) = A(x, y)exp{i [z(x, y) + PrDif/(x, y)J} (7.2.16)

Flow diagram of the computation is shown in Fig. 7.16. Fig. 7-17 illustrates
results of reconstruction of a hologram synthesized in this way for a
unifonnly pained conic surface. As one can see, the hologram is not unifonn.
Since the object scatters most of radiation perpendicularly to its surface,
most of the hologram energy is concentrated along a circle. When fragments
of the hologram along the circle are reconstructed, that is when hologram is
viewed through windows placed along the circle, as in Fig. 7-17, c,
corresponding different aspects of the cone are seen. When viewing window
is situated out of the circle, practically nothing can be seen. Fig. 7-18
provides yet another example of reconstruction of a programmed diffuser
hologram synthesized for an object in a form of a hemisphere on which
numbers 0 and 2 are painted on its four different sides.

r----------I
3-0 I
Object I
object's
renectivity Object's wavetield complex
model
distribution ~ amplitude
-+ FIT
I
A(x,y)exp(i2lt(z(x,y)+PrDitT(x,y»]

Object 3-D ~
sbape z(x,y) I
I
I
t PrDitT(x,y) ~,

Object's I ProgrammabJe diffuser Hologram

surface 4 generator
I
directivity I
pattern I
I
__________ J

Figure 7-16. Flow diagram of synthesis of programmed diffuser holograms.

304 Chapter 7

Figure 7-17. Illustration of reconstruction of holograms with programmed diffuser: Cone

phase profile ofthe object (a), its programmable diffuser Fourier hologram (b), nine hologram
reconstruction windows (c) and corresponding reconstructed images (d) that show views of
the object from corresponding 9 hologram windows

Figure 7-18. Another example of images reconstructed from a programmed diffuser

hologram: an object in form of a hemisphere with numbers 0 and 2 on its four sides as it is
reconstructed trom corresponding four corners (top left, top right, bottom left and bottom
right) ofits hologram.
 7. STATISTICAL COMPUTATION METHODS AND ALGORITHMS 305

7.3 STATISTICAL (MON TE CARLO) SIMULATION.

CASE STUDY: SPECKLE NOISE PHENOMENA
IN COHERENT IMAGING AND DIGITAL
HOLOGRAPHY
Iterative method for generating correlated phase masks described in Sect.
7.2.4 belongs to a family of methods known as Monte Carlo methods.
Monte Carlo methods represent apower tool for optimization, simulation
and testing systems and processing algorithms. They assume working with
multiple realizations of pseudo-random numbers or arrays of pseudo-random
numbers and evaluating the results on average over those realizations. In this
section we will provide a more extended illustration of Monte Carlo methods
using as an example studying statistical properties of speckle noise in
coherent imaging such as ultrasound imaging, synthetic aperture radars and
digital holographie imaging methods.
Speckle noise imposes fundamental limitation on image quality in
coherent radiation based imaging and metrology systems. Appearance of the
speckle noise is associated with properties of physical objects to diffusely
scatter irradiation and with the fact that holograms are, in reality (contrary to
the name "holography" which assumes recording whole wave field),
incapable of distortion less recording the object whole wave field. In
recording the wave field, a number of signal distortions inevitably occur
owing to technical limitations inherent in recording materials and hologram
sensors. These are limitation of the hologram area, limitation of the
recording media or sensor's dynamic range and hologram signal quantization
needed for digital reconstruction ofholograms, to name a few.
In most applications, diffuse properties of objects can be treated
statistically. Statistical theory of speckle noise was developed with regard to
only limited resolution power of coherent imaging devices, or, equivalently,
to limitation of hologram area ([7], see also Sect. 2.8.2). 1t is based on an
assumption that the spread of the point spread function of the imaging
system is much broader then the correlation interval of microscopic
variations of object surface that cause diffuse scattering of irradiation. This
theory is, therefore, valid on]y asymptotically as much as central limit
theorem of the probability theory can be applied.
In many cases, especially in acoustic and microwave holography this
asymptotic assumption is not always applicable. Moreover, in treating
speckle noise problem one should also consider sources of the recorded
wave front (hologram) deterioration other then limitation of the hologram
area, for instance, such as those mentioned above. Analytical treatment of
the relationship between such hologram distortions and statistical
306 Chapter 7

characteristics of speckle noise caused by these distortions is most frequently

very problematic. A solution may be found by means of Monte Carlo
simulation.

7.3.1 Computer model

A flow diagram of a Monte Carlo computer model for studying speckle

noise phenomena is shown in Fig.7-15. In what follows we present, as an
illustration of the model application, some simulation results of hologram
reconstruction of an object with an arbitrary wave front intensity distribution
(image brightness) and a rough surface that is modeled by arrays ofpseudo-
random uncorrelated sampies of the phase uniformly distributed in the range
(-1t,+1t).

....
Generating 2-D array of
pseudo-random numbers
that specify the phase
---. Computing
object's wave
front
Simulating wave
front propagation
IDFT. DFrTl
component of the object
wave front ~~
-~
~ Introducing signal
1- - - - - - - - - - - distortions:
1 2-D array that specifies : - Array size
1 amplitude component of 1 limitation
1 the object wave front 1 - Dynamic
------'l---- range
limitation
Yes - Quantization

•
Comparing
reconstructed and initial
f , wave fronts; computing
( Continue \ and accumulating noise Simulating wave
iterations J.... statistical parameters ~ front
\. ) reconstruction

No .... •
Output data I ...

Figure 7-19. Flow diagram ofa computer Monte Carlo model for studying speckle noise
phenomena

Influence of three types of the wave front distortions on the speckle noise
variance will be illustrated on the base of simulation results ([8]):
limitation of the wave front area;
 7. STATISTICAL COMPUTATION METHODS AND ALGORITHMS 307

limitation of the dynamic range of wave front orthogonal

components, i.e., real and imaginary parts of the corresponding
complex numbers;
quantization of the orthogonal components.
Limitation of the wave front area is implemented in the computer model
via zeroing array of numbers {y r,s} , r, s = 0,1, •••, N - 1, that represent the
wave front. It is specified in terms of the fraction of the array area used for
reconstruction to the entire array area N 2 •
Limitation of the hologram dynamic range is specified in the simulation
in units Thr of standard deviationcr of real and imaginary parts of wave
front orthogonal components ft !:)}
and !~;)} as ft
i(re"",)
. ={y(re,im)
',B ,
'if~,(re,im)I<Thr.cr
11 "B -
(7.3.1)
r,s s;gn{y !~:'im)). Thr . cr, otherw;se

where sign(.) is the signum-function:

sign(y )=y /~I, (7.3.2)

Quantization of the wave front orthogonal components is carried out in

the simulation in the range [- 40" ,40" ] on a nonlinear sc ale defined as (P-th
law quantization)

(7.3.3)

where {y!:,im)}_ are wave front orthogonal components after their limitation
within the dynamic range [- 40" ,40" ]; roundO is an operation of rounding
to the nearest integer, Q -is the number of quantization levels assigned for
quantization within positive/negative halves of the dynamic range [0,40"].
Fig. 7-16 shows examples of images with speckle noise generated using the
described model.

7.3.2 Simulation results

Fig. 7-17 shows simulation results obtained for limitation of area of the
wave field area. The upper band in the image on Fig. 7-17 a) is the original
distribution of the object wave front magnitude generated in a form of a step
wedge function. Lower bands show magnitudes of images reconstructed
308 Chapter 7

from the wave front after limiting its area by portions of 1I8-th of the entire
area. They c1early demonstrate appearance of the speckle noise and the fact
that the higher is area limitation the more correlated is "~~;,", .....l~

c) d)

Figure 7-2 0 . Illustrative examples of simulated images: a) - original image; b) - image

reconstructed in far diffraction zone from 0.9 of area of the wave front; c) - image
reconstructed in far diffraction zone from 0.5 of area of the wave front; d) - image
reconstructed in far diffraction zone after limitation ofthe wave front orthogonal components
in the range ±(J" •

Graphs in Fig. 7.17 b) provide quantitative information regarding speckle

contrast as a function of the area limitation degree (for the definition of
speckle contrast see Sect. 2.8.2, Eq. 2.8.33). The graphs are plotted for
different magnitudes of the object wave front. They show first of all that
speckle contrast does not, statistically, depend on the magnitude ofthe object
wave front: curves for different magnitudes practically overlap. This
confirms that speckle noise that is caused by the limitation of the wave front
area is multiplicative with respect to magnitude of the reconstructed wave
front. The graphs show also that speckle contrast asymptotically, for severe
limitation, is equal to one as it is predicted by the theory (see Sect. 2.8.2).
 7. STATISTICAL COMPUTATION METHODS AND ALGORITHMS 309

However, for moderate limitation, speckle contrast is lower then one and it
decreases with the increase of the fraction of the wave front used for object
reconstruction. Quite naturally, speckle disappears if image is reconstructed
from the whole wave field, which is the case of infinitely high resolving
power of the imaging system.

0.8

0.6
GrLv1l8
218
0.4 3'8
4/8
&8
0.2 &'8
7/8
ß/8
o'~~~~~~~--~~~
\
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Frajjcn dtdqJ;m .....

a) b)
Figure 7-21. Limitation ofwave front area: a) reconstructed images, b) - speckle contrast as a
function ofthe area limitation degree

Measurements of speckle noise distribution density confirm these

observations. They reveal that while for severe limitation of the wave front
area the distribution density P{n) of speckle noise va lues n is the theoretical
exponential distribution:

(7.3.4)

For sm aller limitation degree, when the above-mentioned theoreticallimiting

assumptions are not applicable, it degenerates to the Rayleigh distribution:

n (n2cr"
P(n)=-z exp --z
2cr"
Z
) (7.3.5)

and then, for very smalllimitation, even to the normal distribution.

310 Chapter 7

(7.3.6)

, . . '..... . : .
. ~:::::::::-r""
. -' -' ..... '..
',L: --~ -.. -_.- -
: f I , I •
\-, ........
~~:~~~~~:~~~:~
........... r- .. -. .. .. ,- -

<- . ; . .
~ • , I , •

.. r\.. ,,, .. ,- .. -, ..... ,- . .

. '\....... '.... ,.' ..... '- .. ~ f- .: .... ; .
1\ 1-," ..
! .. t .... I " .. .... ..

.. .. '.. . ~ I .. \-, ....

" • I I I

... -~.. .. '- ... ..' Of1! r .... t " .. , .... I

~J._-="''___
'' ~''_''''''''''''''_'''''_''''''''''
!I) CI :m tSI 2m J)
..................
CI
_.&._
$I
....................210
:IX)
..

a) b) e)

Figure 7-22. Distribution density funetions of speekle noise eaused by the limitation of the
wave front area: a) - exponential distribution, high limitation (30%); b) - Rayleigh
distribution; "Medium" limitation (90%); c) - Normal distribution, very low limitation (97%).
Oscillating eurves show empirie al distribution funetions, plane curves - theoretieal models of
Eqs. 4-6, respeetively.

Spe<:kle S1Dev""rsU$ number 01 quant. lelels. P'0.5. Ni.... ,

"
GrLv
7/8
"
6/8
5/8
4/8
3/8
218
1/8

0.04

0.02

10 20 30 40 50 60
Number of qualIliza U"" la""l$

Figure 7-23. Speekle noise eaused by quantization of orthogonal eomponents

Simulation of the limitation of wave front orthogonal components shows

that, contrary to the limitation of wave front area, limitation of the hologram
dynamic range and quantization result in speckle noise whose statistical
relationship with the hologram signal is of more involved nature. It can not
be uniquely characterized as being additive or multiplicative. In particular,
simulation of quantization of the wave front orthogonal components reveals
7. STATISTICAL COMPUTATION METHODSANDALGORITHMS 311

that the model of additive Gaussian noise can be used with respect to the
magnitude ofthe reconstructed wave front (square root ofthe intensity):

(7.3.7)

where I is intensity of the reconstructed wave front, I its mean value, n is

uncorrelated random process with normal distribution function that models
speckle in the reconstructed wave front and cr n is standard deviation of the
speckle that is inversely proportional to the number 0 quantization levels Q :

1
cr oc- (7.3.8)
n Q'

The simulation allows also to determine the optimal value of the

parameter P of non-uniform P-th low quantization of wave front orthogonal
components (Eq. 7.3.2) that minimizes standard deviation of the speckle
noise. Bar diagrams in Fig. 7-20 show that this optimum lies in interval
P=0.3 +0.5 .

0.2

0. 15

0.1

0.05

33 49 65 81 97 113 129
Number of quantization levels

Figure 7-24. Optimization ofnonlinearity index P in P-th low quantization ofwave front
orthogonal components.
312 Chapter 7

References
1. P. J. Huber, Robust Statistics, John Wiley and Sons, N.Y., 1981
2. W. H. Press, S. A. Teukolsky, W. T. Vetterling, B. P. Flannery, Numerical Receipes in
C. The Art of Scientific Computing, Second Edition, Cambridge University Press, 1995
3. L. Yaroslavsky, M. Eden, Fundamentals ofDigital Optics, Birkhauser, Boston, 1995
4. L. B. Lesern, P. M. Hirsch, J. A. Jordan, Kinoform, IBM Joum. Res. Dev., 13, 150, 1969
5. L. P. Yaroslavskii, N. S. Merzlyakov, Methods of Digital Holography, Consultant
Bureau, N.Y., 1980
6. N. C. Gallagher, B. Liu, Method for Computing Kinoforms that Reduces Image
Reconstruction Error, Applied Optics, v. 12, No. 10, Oct. 1973, p. 2328
7. Goodman, lW.,. Statistical Properties of Laser Speckle Patterns In Laser Speckle and
Related Phenomena, J.C. Dainty, ed. Springer Verlag, Berlin, 1975
8. L. Yaroslavsky, A. Shefler, Statistical characterization of speckle noise in coherent
imaging systems, in: Optical Measurement Systems for Industrial Inspection III, SPIE's
Int. Symposium on Optical Metrology, 23-25 June 2003, Munich, Germany, W. Osten,
K. Creath, M. Kujawinska, Eds., SPIE v. 5144, pp. 175-182
 Chapter 8
SENSOR SIGNAL PERFECTING, IMAGE
RESTORATION, RECONSTRUCTION AND
ENHANCEMENT

8.1 MATHEMATICAL MODELS OF IMAGING

SYSTEMS

Imaging systems always have certain technical !imitations in their design

and implementations and generate images that are not as perfeet as they
would be if there were no implementation limitations. Deviations of real
images from perfect ones may be treated as distortions introduced by
imaging systems to hypothetical perfect, Or " ideal" signals. Correction of
these distortions is the primary goal of image processing.
Methods for distortion correction are based on the canonical model of
imaging systems shown in Fig. 8.1.

Linear Point-wise Stocbastic

transformation non linear transformation
transformation
b(x)

Figure 8-1. A canonical model ofimaging systems

The model represents image formation as a combination of signal linear

transformations, point-wise nonlinear transformations and stochastic
314 Chapter8

transformations that are applied to a hypothetical perfect signal a(x) and

jointly determine the system's output signal b(x).
Linear transformation are specified in terms of the system point spread
function or frequency response. Frequency response of the ideal imaging
system is assumed to be uniform for a11 frequencies in a certain base band.
Frequency responses of real imaging usua11y more or less rapidly decay on
high frequencies. This results in image distortions such as image blur.
Point-wise nonlinear are specified in terms of the system transfer
functions. It is assumed that the ideal system has a linear transfer function.
Deviations of system transfer functions form linear one cause distortions of
gray scale nonlinear distortions.
Stochastic transformations model signal random distortions and cause
random interferences, or noise in output images. Stochastic transformations
are specified by statistical noise models described in Sect. 2.8.
The processing goal is estimating the perfect signal a(x) given distorted
signal b(x) produced by the system. This problem is frequently referred to
as the inverse problem. Signal transformations that are applied to system's
output signals {b(x)} to produce an estimate a(x) of the "ideal" signal are
ca11ed signal recovery or, in application to image processing, image
restoration. Similar problem is image reconstruction. This term is usually
refers to image formation in transform domain imaging such as tomography
and holography.
If the system does not introduce any random distortions and system' s
parameters such as point spread function and transfer function are known,
the inverse problem has a trivial solution: signal b(x) has to be subjected to
transformations inverse to those introduced by the system.
However, one can not, in general, neglect random or similar uncontrolled
distortions such as round-off errors in digital processing. In reality, applying
inverse transformation may result in artifacts that may even be useless. To
solving the inverse problem in such cases, the statistical approach that
explicitly accounts for signal random distortions appears to be the most
appropriate.
Generally, signal recovery, image restoration and reconstruction is
indivisible procedure that should account for and correct all distortions.
However, in practice this process is divided into separate steps carried out in
the order inverse to that distortion factors have in the model.
 8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 315
RECONSTRUCTION AND ENHANCEMENT

8.2 LINEAR FILTERS FOR IMAGE RESTORATION.

8.2.1 Transform domain MSE optimal scalar Wiener Filters

In this section we consider image restoration for a reduced imaging

system model that disregards point-wise nonlinear transformation in imaging
systems and treats image distortions as a combination of distortions caused
by linear filtering and of those caused by action of random interferences.
The design of the optimal restoration procedure requires specifying a
criterion for evaluating the restoration quality. Let {bk} be a set of N signal
sampies (k = 0,1,•••, N - 1) at the output of the imaging system, {a k} be a
set of the system's input signal sampIes that model perfect, or "ideal" signal
and {a k } be a set ofrestored signal sampIes. For the sake of generality, we
will consider the set {a k} as a realization taken from a statistical ensemble
0A or data base of input signals and the set {bk} as a realization of a signal
ensemble generated by ensembles 0A and ON of input signals and of
random interferences.
Define the restoration procedure performance measure as a squared
difference between restored and perfect signals averaged over the available
set of signal sampIes and over statistical ensembles 0A and ON' We will
call this measure mean squared restoration error (MSE). The restoration
procedure R {bk} = {a k } that minimizes this difference:

(8.2.1)

will be referred to as to MSE-optimaljiltering.

For the implementation of the MSE-optimal filtering, we will restrict
ourselves to linear filtering. In general, linear filtering of a discrete signal
may be described as multiplication of a vector of input signal sampies
B = {bk} by a filter matrix H (vector filter, see eh. 4, Eq. 4.1.7):

A=H·B, (8.2.2)

where A= {a k } is a vector offilter output signal sampIes

For signals of N sampies, a general vector filter matrix H has
dimensions N x N . Specification of such a filter requires determining N 2
316 Chapter8

filter coefficients, and the filtering itself requires performing N 1 operations

per N signal sampies. In image processing, the computational complexity
of both determination of filter coefficients and of the filtering may be
become too high because of high dimensionality of image arrays. Fast
transforms that may be computed for O(NlogN) operations (see eh. 6)
allow to radically decrease the filter design and the implementation
complexity. Therefore in what follows we will consider only scalar filtering
in a domain of orthogonal transforms that can be computed with fast
algorithms. This c1ass of filters may be described by the equation:

(8.2.3)

where, T and T- 1 are, correspondingly, direct and inverse orthogonal

transforms, H = diag~, }, (r = 0,1,..•, N -1) is a diagonal filter matrix.
Such a scalar filtering implies the following relationship between filter
output and input signal sampies ~,} = T . A and {ß,} = T· B :

cl, =11,ß,. (8.2.4)

In the assumption of orthogonality of the transform T, one can, by virtue

of the Parceval relationship (Eq. 2.1.28), modify the filter optimality
condition defined by Eq. 8.2.1 in the following way:

(8.2.5)

By computing derivatives over sought variables and equaling them to zero,

one can obtain from Eq. 8.2.5 that optimal scalar filter coefficients may be
found as cross-correlation coefficients between spectral coefficients ß, and
u, of the input and perfect signals:

AVnA AVnN (u,ß:)

11, = AVnA AVn J/ß,/2). (8.2.6)

We will refer to MSE optimal linear filters and, in particular, to MSE

optimal scalar filters defined by Eq. 8.2.6 as to Wiener filters. This name
 8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 317
RECONSTRUCTION AND ENHANCEMENT

gives a credit to Norbert Wiener for his pioneer works in the theory of
statistical methods of signal restoration ([ 1])1 .
In order to implement optimal scalar Wiener filter, one should therefore
know cross-corre1ation {AVnA AVnJa,ß;)} between filter input signal and

perfect signal spectral coefficients and power spectrum ~VnA AVnN ~ß, r)}
of the input signal in the selected basis. The statistical approach we adopted
that assumes averaging of the restoration error over statistical ensembles of
perfect signals and of filter input signals implies that these statistical
parameters should be measured in advance for these ensembles or over the
data bases.

8.2.2 Empirical Wien er filters for image denoising

Consider a ASIN-model (Sect. 2.8.1) in which filter input signal

sampIes {bk} are obtained as a sum of perfect signal sampIes {ak} and
sampIes {nk }of signal independent zero mean random noise:

(8.2.7)

In spectral domain, the same relationship holds for signal and noise spectral
coefficients:

ß,=a,+v" (8.2.8)

where ~,} = T {nk } • For this model one can obtain that

and

AVnA AVnN ~ß,ll)= AVnA AVnN [(0., +v ,Xa; +v; )]=

AVnA (rJ)+ AVnA ~ J) (8.2.10)

1 It will be just to give also a credit to Andrey N. Kolmogorov who developed the similar
theory for discrete signals ([2]).
318 Chapter8

beeause for zero mean noise A VON ~ ; ) = A VON ~ , ) = o. Therefore sealar

Wiener filter for suppressing additive signal independent noise is defined
through its eoeffieients {!,,} as

(8.2.11)

One ean give to this formula a clear physieal interpretation. Define signal-
to-noise ratio as:

A V. I-v ,1 2
SNR = _....:O""A+jU'-'-'::-t (8.2.12)
, AVON IVk., 1
2

Then obtain:

SNR,
(8.2.13)
11, == 1 + SNR,

whieh means that seal ar Wiener filter weight eoefficients are defined, for
eaeh signal speetral eoefficient, by the signal-to-noise ratio for this
eoeffieient. The lower is signal-to-noise ratio for a partieular signal speetral
eomponent, the lower will be the eontribution of this eomponent to the filter
output signal.
In order to implement sealar Wiener filter one have to measure in advanee
power speetra A V0A (r, 2
1 ) and A VON (~ ,
2
1 ) of perfeet signals and of noise
in the seleeted basis. Noise power speetrum may be known from the
specifieation eertifieate of the imaging deviee. Otherwise it may be
measured in noisy input signals using methods deseribed in Seet 7.1.2. As
for the prefeet signal power speetrum, it is most frequently not known.
However, one ean, using Eq. 8.2.10, attempt to estimate it from the power
speetrum A V0A A VON (I ß, 1
2
) of input noisy signals as

(8.2.14)

The latter has to be estimated from the observed signal speetrum 1ß, 12 •
Denote this estimate as IßJ . This empirieal estimate made by averaging
8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 319
RECONSTRUCTION AND ENHANCEMENT

over available realization of input images may, when used as areplacement

for AVUA AVUN (Ißl), give negative values for some spectral coefficients
because of the limited depth of the averaging. Since power spectra can not
assume negative values, the following modified spectrum estimation may be
adopted:

(8.2.15)

In this way we arrive at the filter:

(8.2.16)

We will refer to this filter as to the empirical Wiener filter.

If the imaging system noise is known to be white noise with variance 0'; ,
the empirical Wiener filter takes the form:

11, = [
max 1-ß, 1
2

Iß ,1
-O'n

2
2

; ]
0 . (8.2.17)

As a zero order approximation to the input images power spectrum, power

spectrum 1ß,1 2 of a single input image subjected to filtering may be used. In
this case empirical Wiener filter weight coefficients are found as:

11
,
= max[Iß,I Z -0';.
IßJ'
0]. (8.2.18)

Note that such an empirical Wiener filter is adaptive because its weight
coefficients depend on the spectrum of the image to which it will be applied.
Weight coefficients ofscalar Wiener filters assume values in the range
between zero and one. Aversion of the empirical Wiener filter of Eq. 8.2.18
with binary weight coefficients:

11 =
, 0,
if{t, Ißl
~ Thr
otherwise
(8.2.19)
320 Chapter8

where Thr is a rejecting threshold is called the rejecting filter. As it follows

from Eq. 8.2.18, the rejecting threshold has a value of the order of magnitude
of the noise variance cr ~ . Rejecting filters eliminate from the input images
spectra all components for which signal-to-noise ratio is lower then a certain
threshold.
Aversion of the empirical Wiener filter and of the rejecting filters that are
implemented with wavelet transform image decomposition is known as
wavelet shrinkage filtering ([3]). Empirical Wiener filtering according to Eq.
8.2.18 an a wavelet jargon is called "soft thresholding". Rejecting filtering
according to Eq. 8.2.19 is called "hard thresholding". Fig. 8-2 shows flow
diagram of signal denoising by the wavelet shrinkage

Output
Input
Softlhard
thresholding

.L()w pass
flltering and
downsampling

SofUbard
thresholding
Low pass
fiJtering and
downsampling

Lowpass
illtering and
downsampling

Figure 8-2. Wavelet shrinkage: signal denoising in wavelet transforrn domain

Described empirical Wiener filters for signal denoising are particularly

very efficient if signal andlor noise spectra are weil concentrated and are
separated in the transform domain. A typical example of such a situation is
filtering of narrow band noise, whose spectrum has only a few components
in the transform domain. Figs. 8-3 through 8-5 illustrate examples of such a
8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 321
RECONSTRUCTION AND ENHANCEMENT

narrow band noise filtering. Fig. 8-3 demonstrates filtering periodical noise
pattern in an image. Such interferences frequently appear in images digitized
by frame grabbers from analog video. Left column in Fig. 8.3 shows input
and filtered images. Right column shows averaged spectra of input and
output image rows. One can c1early see anomalous peaks of noise spectrum
in input image spectrum that are eliminated in the output image spectrum
after applying empirical Wiener filtering designed using methods of noise
spectrum estimation described in Sect. 7.1.3.2. Note that the fiItering is
carried out in this example as 1-0 row-wise filtering in OFT domain. This
type of interferences can also be successfully filtered in Walsh transform
domain ([4]).

+. . . . . . . ,. . . . . . . . ,. . .
Column wise avera ed ower spectrum alon TOWS
14 ................ !................... j.................. j................

12 ········..··.1·················1...··········.1 .. ·········

10 .\............. ···············t·················:··········..·····!.. · ........ ......,....

8 ·t·······'·····r""'·········· .. ·r--········-----.. ~
. f ~ : l
!'--------·------·-~···················i················-·1'---'

6 .. ~t~JCC==1\=J~A.( · ·'· ·.
20 40 60 80 100 120

"Filtered"
14 ...............,.................................. , ............... , ...............•................. , ..... .

12 ··························;······,,·,,·'·'·'1'···············r··············T····

180 (:::.. ::::............:::::.. ::.::: .. ::::::. :::::::: ..":':::::::::::::::::"'::"':.:1:':.:":'..:':'::1:::::'

6 '~~~=~~::J:::=~:::r""""""'T'
20 40 60 80 100 120

Figure 8-3. Filtering periodic interferences

Fig. 8-4 iIIustrates similar 2-D rejecting filtering for eliminating

periodical noise components in a digitized Fresnel hologram. Fig. 8-5 shows
yet another example of empirical Wiener filtering narrow-band interferences.
Banding noise in the initial image shown in Fig. 8-5 is characteristic for
imaging systems with mechanical scanning. This particular image was
produced by an atomic force microscope. Banding noise randomly changes
image dc component (row-wise mean value) in the direction of scanning.
322 Chapter8

Oigitized hologram OFT spectrum ofthe Pattern of 2-D rejecting

hologram fil ter coefficients (black for
land white for 0)

Figure 8-4. Periodical noise in a Fresnel hologram and a pattern of rejecting filter coefficients
for removing the noise. The noise manifests itself in horizontal and vertocal bands in OFT
spectrum of the hologram. (The hologram was kindly granted by Or. T. Kraus, BIAS,
Germany)

Figure 8-5. Filtering banding noise

Upper right plot in Fig. 8-5 shows row-wise mean values, or row-wise
image Radon transform, (horizontal coordinate) as a function of the row
number (vertical coordinate). Taking as an es ti mate of perfect mean row-
 8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 323
RECONSTRUCTION AND ENHANCEMENT

wise values a mean value over all rows (shown with a straight line in the
bottom right plot in Fig. 8-5), one can estimate banding interference on
every row by subtracting this estimate from the observed mean values.
Subtracting the found values from all pixels in the corresponding row
eliminates the noise.
Wiener filtering of wide band noise and especially white noise is less
efficient. When filtering of white noise, Wiener filter tends to weaken low
energy signal spectral components. However usually these components are
exact1y the components that are the most important because they carry
information about signal changes such as at edges in images. Moreover,
Wiener filtering converts input white noise into output correlated noise
though with a reduced variance. As one can see from Eq. 8.2.11, in case of
the intensive input white noise, power spectrum of the residual noise is
roughly proportional to the signal power spectrum which means that the
residual noise becomes, statistically, signal alike. This may hamper
subsequent image analysis. In particular, it is well known that human vision
is more sensitive to correlated noise that to white noise ofthe same intensity.
Therefore Wiener filtering for image denoising may even worsen images.
Fig. 8-6 gives an example of computer simulation of Wiener filtering of a
test image with additive white noise. In the simulation, one can implement
the ideal Wiener filter because both signal and noise spectra are known. The
empirical Wiener filter was implemented in this example according to Eq.
8.2.18. As one can see from the figure, ideal Wiener filtering does improve
image quality. For the empirical Wiener filter, improvement is much less
appreciable. The difference image between initial noisy image and the result
of the empirical Wiener filtering (restoration error) shows that the filtering,
along with noise suppression, destroys image edges.

8.2.3 Image deblurring, inverse filters and aperture correction.

Consider now an imaging system model that accounts also for signal linear
transformations in imaging systems. Suppose that the linear transformation
unit can be modeled as a scalar filter with filter coefficients {A. r j in a
selected basis. For DFT basis, this assumption is just, to the accuracy of
boundary effects, for shift invariant linear filtering (see Ch. 5). In this case
coefficients {A. r j are sampies of imaging system frequency response. In
ideal imaging systems, they should all be equal to unity. In reality, they
decay with the frequency index r, which results, in particular, in image blur.
Processing aimed at correcting this type of distortions is frequently referred
to as image deblurring.
324 Chapter8

·i,~'iik~· )'~~:.\r~:~ ·t.f.~ :'$.c.r;~·; ).~ : . .~: , lrule reoo,~1)' uni. 111011: 8~'I1I), alt•
. . ~mit. 'i~(U, : .2.~': · jti~ ·( ~~~rp .· · ; 'oplerlLf lhw. an:: .!\Iang J be rnon Ilmulll'l\
IJl! . '}:~;:-i!~~ J:'1tt;~:. k.o\:~~!); ~\"J,P.ff~, ~ !' I.ey :iJtv'.lh' ~ evsY kJl~\jo1l SCAlt-ftom II
~ ~':iioi~ß CU! " !:~~'';•• <~(j;, ";" ~l'led ~. :lenniniog IM $'Inlt!tUJ'C of unroolved s:
~ . . ...... . t ;~~ .r..\ ~ ...... ''/ :'~.'" .,~ _" "', .
c.L I!l:: .,~$st ~~!I !t:(l .:_::il\\'t)!(1 i)\. ;11,
',.1 . . .

~~~ eI11 .o f ltu: ticies. mol«u 1.,;1 . SI8teO i 11 lt!:

;~.~~.y1!r.".ijlte~·f:( ~lpcij lik~:'~~~.d\;". alPet? pcobltntu de<..crlbcd 1U.:~ lbl5: OJ\'
~,~ft.':i-?~~·1. . :';6(1~k~j:. ·',,!,~i~:~:;{~~ iII Cpod .....-ed § . U n t Cll'IUClliiM,7, ,."bl:II SllUOO
(~:-i- . ~ ~-?:J!~~. ; .:t5:;;:M, 'i!~f1. :fft% . oreC:&Il' bt a.id , JIQ'II' iL' ;-tbted IfJjl ls t
)j; '~E ~.!F.idt":f~'~' ,;.";!.~: c1,U'mat;;,t ~f' n. mllg be usr.11 t D fllmis!! :m e:mrruu~ -ui
. fl:~.itA·c.~~\,t~·k-'i~ ~ r~ ~&it' : • t(. g illl:Orniple4 by. noi$c. ~ d·~ .lloitl! 11
:.. ·flf.: ,; Äi6f."~q~~~h}a '('Ci~J~;fI:1)i{l:jil;· IC. (',I\n'1I;1; .melJorrue ,he elfeas oCIlle no
l~,:q. ciif~ Atti.~~~: .i!;~i;. ;ilii~' lf ,~..,.Ü~;:; loe r.ldicaJ ch~ in-f! liw:u i ~ wliql
gC'l:~tE :.. f-::~,~~)ji~_"::~Ji1,.f.1'; ~ß,::,~ ~(;. 80rithnt f~ coltlputillS]itom.{! \"'lu.! ADO
•~~·Ei.~Jt ·.ni~~~lelti h';'~r:N!;*d f;(!N~ m1 .C:.'a'R il be'uodully inCOmOr3!e:lm OlJl
)Ct.,,~,:(,&.n,:f.i)!tih- ' ~ ~~".i~. ~j~'·O: ;~~J~ Thte:e (aDd ot~> U~ litt kI nds oEqUC!.1
. ~ ~. ..,; t, ~: ;~,~: '.~ ~ , .. ~'.: ~~_-~.. . ;,~ . ~.:.f;:;:\~;;,~"'~ .,:,: '" - - . • • -Sl'" ... .al. .. . , : •• • L. _. ____ ••• • • I. J. "' ... ....
Test noisy image
1111' ~ 1'~Qi.·,!r~ 5.ila. I U~t' lH:llcU!J), :JjlO
.,.,l!I"",,' rhJlI UI.! ItiUilfW ·thr 1ll0M 1I11lll!lll1
IU'Y j:tr,'OlV e ~'t..t'V lJl''''lI .1X:cle:...oi rmn I
~.\:Imil1in " bi: ::Inl'::tlu.~ er., rmri:iolve-d $ ,
GlI nl nlC ntllCSl J1lOlc:u1tn . ' ;'lifled in ie
.;.0 ~ry jltt3bleJu. '" dtMrlbcd l!i:t .t:~: {)j ' et
,-1 1 ~JT",J u;::ed Il. un.llt'ur..lHt:h , ~III!U SlllleC
Qa'. .:Ir" bI! !:Cd. 110".1L': 1'ri:ud trl -J1 l:;. ~
•• ,n:.sN~.c!V dln. )llll li rlßl :"llJlTUuc tui
~ I' R i\; ~~ct b+t l1Oiu:, ~ U:r. swt.w .
e, rJlII ; ~C lIJt)di~till! flie~H pf tbe "'0
l ce r,ltib tal ct.'OIIUIU Ill' livcll -ij .! 'lHur
gcr,tllOi tor. o.uI1PUJI~_:. fruln.j~ \Vtw.~~ n .
I'
1[1' Gm JI b.: o!:''!·i u:ly in,,'O ~()tl:,:lIed 1II Illir

·nt~.., iiUltl qlht"iAt ~t:t ti :~ y.!J1tis 0:- t'\'tr.1

... _ ~I I P" r.1k ai r Jr P .... .
· ' , . 9,!' .... . ~ . "t · ,~ h h~· .
Empirical Wiener filtered image

Figure 8-6. Ideal Wiener and Empirical Wiener filtering for image denoising

For such systems, we have in the transfonn domain :

(8.2.20)

Using Eqs, 8.2.6, one can obtain that the scalar Wiener image restoration
filter is defined in this case by the equation:

SNR 1
(8,2.21)
11 = ~ l+S;Rr
r '

where SNR is signal-to-noise ratio at the output of the linear filter unit of
the imaging system model:
 8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 325
RECONSTRUCTION AND ENHANCEMENT

(8.2.22)

Correspondingly, the general empirical Wiener filter, empirical Wiener filter

with zero order approximation to perfect signal spectrum and the rejecting
filter for aperture correcting and image deblurring will be in this case as
folIows:

(8.2.23)

(8.2.24)

and

11 r _{~'ifIßJ
- Ar ~ Thr . (8.2.25)
0, otherwise

All these filters may be treated as two filters in cascade: the filter with
coefficients

in. 1
11 r = -:;: (8.2.26)
r

usually called the inverse filter and signal denoising filters described by Eqs.
8.2.17 - 19. Inverse filters compensate weakening signal frequency
components in the imaging system while denoising filters prevent from
excessive amphfication of noise and perform what is called
"regularization" of inverse filters. As one can see from Eq. 8.2.22, weight
coefficients of denoising filters for small {A r } fall faster than weight
coefficients ofthe inverse filter grow.
One of thc most immediate applications of inverse filters is correcting
distortions caused by finite size of apertures of image sensors, image
discretization devices and image displays. We will refer to this processing as
to aperture correction.
 326 Chapter8

Let an image sensor and discretization device is an array of light sensitive

elements with a square aperture of size d(d) x d(d) (Fig. 8-7). Then
frequency response of the individual sensor elements is
d(d) / 2 d(d) /2

n(Jx' /y)= fexp(i2n/ x ~ fexp(i2n/ Y~ =

- d(d) /2
x
_d(d) /2
y

• I. f d(d) / 2) sin{n# d(d»)

SID\n x (d) ~ :I y _. I. f d(d»). I. f d(d») (8.2.27)
(d) - SIDC\n x sIDC\n x ,
n/ß /2 rt/yd

or, in dimensionless coordinates {lx = /x/).x,ly = /y/).x}, where /).x is the

discretization interval in both coordinates,
(8.2.28)

I I
: fiel) : I Ax I

~~. : n : n
Sensor's aperture
______ L.. ___ .1 ___ _

Figure 8-7. Arrangement oflight sensitive elements in image sensor arrays

The discrete frequency response is then:

(8.2.29)

where (j(d) = d(d) / /).x .

If the image display device has also a square aperture of size d(r) x d(r) ,
the overall imaging discrete system frequency response is:
'A T,S
= 'Ar 'A s =sinc'n(j(d)'A
~ r
/ N x ). sinc{n(j(r)'A
~ r
/ N x )x
sinc{n(j(d)'A s / N JSinc{n(j(r)'A s / Ny) . (8.2 .30)

Parameters d(d) , d(r)and /).x are imaging system design parameters that
may be know system' s certificate. They can be used for correcting image
distortions by processing images in computer.
8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 327
RECONSTRUCTION AND ENHANCEMENT

Fig. 8-8 illustrates an example of the aperture correction of an air

photograph. Right image in this figure is obtained by applying to the left
image an inverse filter for the system's frequency response defined by Eq.
8.2.20 with (j(d) =(j(r) =1.

Initial and aperture corrected images

Figure 8-8. Aperture correction: initial (Ieft) and aperture corrected (right) images

In synthesis of computer generated holograms, aperture correction is

required to compensate masking images reconstructed from holograms by
the frequency response of the hologram recording device (see Sect. 13.3).
This masking may result in substantial reducing image contrast on a
periphery of image plane. The aperture correction can be implemented by
multiplying object image, before the hologram computation, by the function
inverse to the hologram recording device frequency response. This method is
illustrated in Fig. 8-9.

Figure 8-9. Effect of spatial masking of reconstructed images due to the finite size of the
hologram recording aperture (a) and the result ofits compensation (b) (adopted from [5])
328 Chapter 8

8.3 SLIDING WINDOW TRANSFORM DOMAIN

ADAPTIVE SIGNAL RESTORATION

8.3.1 Local adaptive filtering

In the derivation of the MSE optimal scalar linear filters, the size of input
signal vectors was a free parameter. In the filter implementation, one should
select the fashion in which signals are to be processed. Given an image to be
processed, filtering can be designed and carried out either over the entire set
of available image sampies or fragment-wise. We will refer to the former as
to global jiltering and to the latter as to local filtering.
There is a number of arguments in favor of "local" filtering versus
"global" one:
- "Global" filtering assumes signal "stationarity". For scalar linear
filtering, this means signal homogeneity in terms of signal power spectra:
spectra measured over different signal fragments should not substantially
deviate one from another. One hardly can regard such signals as images as
being spatially homogeneous. F ig. 8-10 illustrates image spatial
inhomogeneity on an example of a noisy "Lena" image. One can see that
while noise in the image is spatially homogeneous, individual fragments of
the image substantially differ one from another visually and, obviously, in
their power spectra as weIl.
- Adaptive filter design assumes empirical evaluation of signal spectra.
Global evaluation of image spectrum disregards spectra variations due to
image inhomogeneity. This obviously results in neglecting local image
information in favor of global one which usually contradicts processing
goals.

Figure 8-10. Noisy image: global versus local analysis

 8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 329
RECONSTRUCTION AND ENHANCEMENT

- Objects to be recognizable visually have to contain sufficiently large

number of resolution cells (pixels). As an immediate illustration of this fact
one can recall that, for the reproduction of printed characters, one needs a
array of 8x8 or more pixels. The same and even to a greater degree holds for
"texture" images. Texture identification is also possible only if texture area
contains sufficiently large numbers of pixels. This means that image can be
regarded as a composition of object domains with the linear size from
several to several tens of resolution cells.
It is weIl known that when viewing image, eye optical axis
permanently jumps "randomly" over the field of view. Human visual acuity
is very uneven over the field of view. The field of view of the human vision
is about 30°. Resolving power of the vision is about one angular minute.
However such a relatively high resolution power is concentrated only within
a small fraction of the field of view that has size of about 2° ([6]). Therefore,
the area of acute vision is ab out 1/15-th of the field of view. For images of
512x512pixels this means window ofroughly 33x33 pixels.
The theoretical framework for local filtering is provided by local criteria
of processing quality introduced in Sect. 2.1.2. For optimallocal MSE scalar
filters, filter coefficients ~!k)} are, according to the local criteria, defined by
the equation:

(8.3.1)

where W is it the filter window size and the size of the area over wh ich
filtering performance is evaluated, k is an index of the window position,
{ß ;k)} are spectral coefficients of the filter window sampIes, fx. !k)} are
spectral coefficients of filter window sampIes of a hypothetical perfect
signal. Solutions obtained in Sect. 8.2.2 and 8.2.3 for empirical Wiener
filters for image restoration may then be extended to local filtering by
replacing in Eqs. 8.2.16 - 19 and 8.2.23-26 signal spectra with corresponding
spectra ofthe signal window sampIes.
The most straightforward way to implement local filtering is to perform it
in a hopping window. This is exactly the way of processing implemented in
most popular audio and image coding methods such as JPEG and MPEG.
"Hopping window" processing being very attractive from the computational
complexity point of view suffers however from "blocking effects" - artifacts
in form of discontinuities at the edges of the hopping window. Obviously, an
ultimate solution of the "blocking effects" problem would be sliding window
processing. Sliding window filtering is local adaptive because, for empirical
330 Chapter8

Wiener filters, filter eoeffieients depend on loeal speetra of image

fragments,.
In the sliding window filtering, one should, for eaeh position of the
filter window, eompute transform eoefficients of the signal vector within
window, design on this base the filter, modify aceordingly transform
eoeffieients and then eompute inverse transform. With sliding window,
inverse transform need not, in principle, be computed for all signal vector
within window since only the central sampie of the window has to be
determined in order to form the output signal in the process of image
scanning with the filter window. Fig. 8-11 illustrates this process.

Polnt-wln InVe1"se
multipUcadon
byÜte mter
Fllter
mask

spednlm

Figure 8-11 . The principle of local adaptive filtering in sliding window

8.3.2 Sliding window transform domain DCT filtering

The selection of orthogonal transforms for the implementation of the

filters is governed by
the required aecuraey of approximation of general linear filtering with
scalar filtering;
the convenience of formulating apriori knowledge regarding image
spectra in the chosen base;
- by the accuracy of the empirical spectrum estimation from the observed
data that is required for the adaptive filter design;
- by the computational complexity of the filter implementation.
 8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 331
RECONSTRUCTION AND ENHANCEMENT

Among all known transforms, Discrete Cosine Transform proved to be

one of the most appropriate ones for sliding window trans form domain
filtering ([7-9]). DCT exhibits good energy compaction capability which is a
key feature for the efficiency of filters. Being advantageous to DFT in terms
of energy compaction capability, DCT can also be regarded as a good
substitute for DFT in signal/image restoration tasks with imaging system
specification in terms of their frequency responses. DCT is suitable for multi
component signal/image processing. The use of DCT in sliding window has
low computational complexity owing to the recursive algorithms for
computing DCT in sliding windows described in Sect. 5.1.3. In addition note
that, if the window size N w is an odd number, the inverse DCT transform of
local spectrum ß;k) for computing window central pixel a k+(N",-I)i2

r=1 .=1

involves only signal spectrum coefficients with even indices. Therefore only
those spectral coefficients have to be computed and the computational
complexity of sliding window filtering in DCT domain is O[(W + 1)/2]
Y
operations for I-D filtering and ol(w + 1 /4 J operations for 2-D filtering
in a square window of W x W pixels.
Fig. 8-12 shows an example of local adaptive empirical Wiener rejecting
filtering in DCT domain according to Eq. (8.2.19) of a noisy
electrocardiogram in the window of 25 sampIes. Upper plot shows initial
electrocardiogram. Image in the second row ofthe figure shows local spectra
of the electrocardiogram for each window position. Image gray levels
represent here amplitudes of spectral coefficients, vertical coordinate
represents spectral component indices, horizontal coordinate represents
window position indices. Such a signal representation is its time-frequency
representation. Image in the third row displays the same spectra after
rejecting signal spectral components that do not exceed the selected
threshold. The bottom plot shows the filtering.
Fig. 8-13 illustrates local adaptive rejecting filtering of a piece-wise
constant test image. Upper left image is th initial test image. Upper right
image is the input image with noise added. Left bottom image is the result of
the filtering. Right bottom image shows a map of the rejecting filter
"transparency" for each window position; i.e., the ratio of the number of
ones in the array of the filter coefficients to the window size. One can see
from this image that, when filter window seats on an image patch where
image gray values are constant, the filter is almost completely opaque and
332 Chapter8

preserves only local dc image component. On the contrary, on the

boundaries of the patches, the filter is almost completely transparent thus
preserving edges of the patches. This feature of the local adaptive filters can
be even more vividly seen from plots shown in Fig. 8-14 of one ofthe image
rows. The edge preserving capability is an important advantage of the local
adaptive filtering comparing to the global filtering.

2500
2000
1500
1000
~. --~~-:Jt :-:::- .
~~~~-;!~.- ~-~~~.~~
Initial signal

::,:---~::..-m-= ~~
500 1000 1500 2000
local OCT speclrum;SzW=25

5
-:------.-------------
10
15
20
25
500 1000 1500 2000
Modified local speclrum: SZW=25: Thr-O.OOl
~ - -- ------ ~ -
5
10
15
20
25
500 1000 1500 2000
Reslo<ed signal
2500

2000

1500

500 1000 1500 2000

Figure 8-12. An example of denoising I-D signal (electrocardiogram)

An example of such a denoising of areal MRI image is shown in

Fig. 8-15. Left image in this figure is the initial image; central image is the
result of the filtering and right image is a difference image between initial
and filtered ones. From the difference image one can see that it looks almost
chaotic and does not contain any visible details of the initial image.
Local adaptive filtering allows also denoising images with signal
dependent noise such as speckle noise. The characteristic feature of the
speckle noise is that its standard deviation is proportional to the signal (see
Sect. 2.8.2). For filtering speckle noise with local adaptive filters, one should
set parameters cr n and Thr of the filters of Eqs. 8.2.17 through 19 to be
proportional to the image local dc component ß~kl. Fig. 8-16 shows an
example of such a denoising of an ultrasound image.
8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 333
RECONSTRUCTION AND ENHANCEMENT

Figure 8-13. Sliding window DCT domain denoising

32-th row of the Initial image

200 F~"'''-'
158 .---- :::::: :: -------------~------I----~=='''''Iiuo L~~
~::::::::::::t:::::: :::: j-" - .:::.::::l::: -::::::::~
50 ~"·--·----------- -- f- --------- ~t·--' --· ........ ±
50 100 150 200 250
32-th row of the nolsy Image

200
100
~=--,"r-Wj~l' (· -------- --1--- -ur:'!~~:----,..~-·--t:·T--~'!:~rtl
0/---- -- - ------------- ,--- .-- -.. - W. __ .. ____ , ._. - .- .. ---,-
o !ui,•.
v. 1/1 ./' w. .. .J..J ~ I........, : i
50 100 150 200 250
32-th ro\IV of the filtered image

200
100 ~"" J'"''j----··---·-··I~-·-···{c·:tj·-·f------
:"·"·"·"-r··"·· . . . :
--- .. --- .. --l~~ -.--·-J··r
·-----·-::1:1
t . __ _ \ ____
Z
1
o _.~. - =-1 • \0-.. ,' •

50 100 150 200 250

Figure 8-14. A typical row ofthe initial, noisy and filtered images
334 Chapter8

Figure 8-15. Denoising MRI image. Left to right: initial noisy image, filtered image, and
differenee between initial and filtered images

Figure 8-16. An example of loeal adaptive filtering speekle noise in ultrasound images: initial
(Ieft) and filtered (right) images

8.3.3 Hybrid DCT /wavelet filtering.

An important practical issue in using local adaptive filters is

selecting the filter window size. This selection is govemed by the
typical size of image details that should be preserved in the filtering.
8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 335
RECONSTRUCTION AND ENHANCEMENT

In general, optimal window size depends on the image and may vary
even within the image. This requires filtering in multiple windows
with an appropriate combination of the filtering results.
One can avoid filtering in multiple windows by combining multi
resolution property of the wavelet shrinkage and local adaptivity of
DCT filtering in the hybrid filtering method ([10]). In the hybrid
filtering hard or soft thresholding in each scale level of the wave let
filtering shown in a flow diagram of Fig. 8.2 is replaced by local
adaptive filtering in DCT domain with a fixed window size. Owing to
multi scale image representation in wave let transforrn, this
replacement imitates parallel filtering of the input image with a set of
windows accordingly to the scales selected. Flow diagram of the
hybrid wave letlDCT filtering is shown in Fig. 8.17.

Output
Input
Sliding
window
filtering
Low pass
filtering and
downsampling

Sliding
window
ftItering
Low pass
tiltering and
down 'ampling

Low pass
filtcring and
downsampHng

Figure 8-17. Flow diagram of the hybrid sliding window DCT domain and wavelet denoising
336 Chapter8

For high resolution images, DCT filtering in moving window requires

window size 3x3, otherwise the filtering may result in loss of tiny image
details. This requirement limits the noise suppression capability of the local
adaptive filtering that increases with the window size. In the hybrid filtering,
when window 3x3 in the first scale is used, in the scale 2 effective window
size is 6x6, in the scale 3 it is 9x9 and so on. This promises an increase to the
filtering noise suppression capability.
In DCT filtering in the window of 3 x 3 pixels, only the following four
basis function are involved:

q> 0,0 =[ 111 1]

1 1 ; q> o,z =~ +2 +
[- 1
2 +2;
- 1 -1]

1 1 1 -1 -1 -1

q> z,o =~ [ - 1
- 1 + 2 - 1]
+ 2 - 1 ; q> z,z = ~
[-1 - 2
-1]
- 2 + 4 -2 . (8.3.3)
-1 +2 -1 -1 -2 -1

The second and the third functions represent what is called vertical and
horizontal Laplacian operators. The last function can be decomposed into a
sum of diagonal Laplacians:

=~l=~ -2:~ =~j=~{l:~ :~ +2=~j+l=~+2 +2

-1
q>z,z
-1 -1 -1 -1 -1
(8.3.4)

Therefore, DCT filtering in the 3 x 3 window may be replaced by

filtering in the domain of four directional Laplacians. Experimental
experience proves that, for high resolution images, such an implementation
is advantageous to simple DCT since it produces less filtering artifacts. Its
use in hybrid filtering promises additional advantages because it is
equivalent to the corresponding increase of number of effective basis
functions. Examples of the effective set, for the input image plane, of these
directional Laplacians in four scales are shown in Fig.8-18.
Results of the experimental comparison noise filtering capabilities of
the sliding window DCT filtering, wavelet shrinkage filtering and of the
hybrid filtering reported in [10] are summarized in Table 8-1. For the
wavelet shrinkage filter, a code of image pyramid from University of
Pennsylvania package [11] was used in the experiments. For all filters,
optimal parameters were experimentally found that minimize MSE between
8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 337
RECONSTRUCTION AND ENHANCEMENT

initial noise free image and filtered ones. For DCr moving window filtering,
window size and parameter Thr were optimized. For wavelet shrinkage and
hybrid filter, parameters Thr were optimized and the best of the following
types of wavelets was chosen: Haar wavelet ("haar"), Binomial coefficient
filter ("binom3", "binom9" and "binomI3), Daubechies wavelet ("daub2"
and "daub4") and Symmetrie Quadrature Mirror Filters ("qmf5", "qmf9"
and "qmfl3"). rhe table summarizes results obtained for four of 12 test
images (a test piece-wise constant image, "Lena" image, an MRI image and
an air photograph) of 256x256 pixels with 256 quantization level and
standard deviation of additive Gaussian noise of 13. For Dcr moving
window filtering, optimal window size is shown in brackets. For the wavelet
shrinkage, the best wavelet filter kerne I is indicated in brackets .

.,
1It:t

Figure 8-18. Effective basis functions in four scales ofthe hybrid DCT/wavelet filtering

Table 8-1. Standard deviation of the difference between initial noise free and filtered
images for different filtering methods Standard deviation ofadditive noise is 13)
Filter Piece-wise Lenna MRI Air photo
constant image
ima2e
DCT Hard 8.1 9.4 6.7 8.2
(3x 3) (3x 3) (5x 5) (3x 3)
Soft 7.5 8.6 6.3 7.7
(3 x 3) (5x 5) (7x 7) (3x 3)
Wave let Hard 8.6 10.1 8.5 9.3
shrinkage. (binom5) (binom5) (binom5) (binom5)
Soft 8.4 9.0 7.8 8.1
(binom5) (amf13) (binom5) (binom5)
Hybrid Hard 8.7 9.4 6.6 8.2
(binom5) (binom5) (binom5) (binom5)
Soft 7.9 8.6 6.2 7.5
(binom5) (binom5) (binom5) (binom5)
338 Chapter8

Fig. 8-19 compares filtering results for two of the test images. To ease
visual comparison, 64x64 pixel image fragments are shown.

Figure 8-19. Sliding window DCT domain, wave let and hybrid filtering for image denoising:
a) initial images; (b) fragments of the noisy images, (c, d) DCT filtering with hard and soft
threshold respectively. (e, f) wavelet shrinkage filtering with hard and soft threshold
respectively. (g, h) hybrid filtering with hard and soft threshold respectively.

In concIusion note that above described sliding window OCT, wavelet

and hybrid filtering methods may be treated as a sub-band decomposition.
Sliding window filtering in OCT domain is filtering in sub-bands uniformly
distributed in the base band with the number of the sub-bands equal to
(W + 1)/2 in 1-0 case. Wavelet shrinkage is filtering in sub-bands arranged
in the base band in a logarithmic scale with the number of sub-bands equal
8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 339
RECONSTRUCTION AND ENHANCEMENT

to the number of the resolution scales. Hybrid filtering combines logarithmic

arrangement of sub-bands of wavelets with uniform arrangement of sub-sub-
bands within wavelet sub-bands. Fig. 8-20 illustrates this interpretation on
an example of wavelet filtering in 4 scales and sliding window filtering in
DCT domain in the window of 3 sampies.

Subbands of b inomS Subbands 01 blnomS&DCT3x3

2.S
1 .~
~
'... .,
'j
; .
! .
1
. : ·1 o.~ ~ ." . "
'!
., .:.

I
0 .4 0 .6 0 .8 1 0 .2 0 .4 0 .6 0 .8 1

i
u u .u ~:~ lf~" _ '
o~ ~: ..
'. '.]
j
::V\',~~ .
0 .2 0 .4 0 .6 0 .8 1 0 .2 0 .4 0 .6 0 .8 1

! ul ~i~..~· . . ·:.· . t . .: ·.:)::··1

0 .2 0 .4 0 .6 0 .8 1 0 .2 0 .4 0 .6 0 .8 1

0 :3
/: :~'::.' ~
8~ ~ . u/ .. • u .
'
'<',.
~
~ • . .. :! . . . . . . .
... , . . .. ~l f·V~
/ ~. ". uj'-1'"'U'~"j
81_/· . : ..... · :-··~ . j . ' . _
0 .2 0 .4 0 .6 0 .8 1 0 .2 0 .4 0 .6 0 .8
u,.. u. u, ' '', ...
o+>;-~
8 ~ U' u ;~ u.

. ;_.-+-~~ .. ·~~l . . . .!'" .

6 . ," ... .. .. ''' , .. , " ..

0 .2 0.4 0 .6 0 .8
!·· 1 1
~
0 .2 0 .4 0 .6 0 .8 1
Normalized 1requency NormaHzed frequency

Figure 8-20. Wave1et domain (Binom5) and hybrid sliding window ocr domain processing
as sub-band decompositions
340 Chapter8

8.4 MULTI-COMPONENT IMAGE RESTORATION.

The above theory of MSE optimal scalar filtering can be extended to

restoration of multi component images such as color and multi spectral
images, video sequences, images of same objects generated by different
imaging devices. Such a processing is sometimes called data fusion.
Consider an M component imaging system and assume that each image
of M components can be described by the model of Eq. (8.2.20):

{ß (m) =A.(mh (m) +v (m)}

r r r r' (8.4.1 )

where m is component index, m =1,2,.•., M , {ß !m)} and tx!m)} are spectral

coefficients of the observed and perfect image components, {A.~m)} are
component linear transformation spectral coefficients and ~ !m)} are additive
noise spectral coefficients in m-th component. Suppose that image
components are restored by a linear combination of scalar filtered input
image components:

(8.4.2)

where ~ !m,/) }are restoration scalar linear filter coefficients.

From the MSE criterion

(8.4.3)

one can obtain the following optimal restoration filter coefficients ([12]):

(8.4.4)

where

(8.4.5)
 8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 341
RECONSTRUCTION AND ENHANCEMENT

are signal-to-noise ratios in image component spectral coefficients.

An important special case of multi component image restoration is
averaging of multiple images of the same object obtained with the same
sensor. The averaging

M
"b(m) /cr (m)
L... k n
(8.4.6)

where {b!m)} are sampies of image components and ~ ~m)} are standard
deviation of uncorrelated additive noise in image components, follows from
Eq. 8.4.2 and 8.4.44 in the assumption that tx~m)=ar}, {A~m)=l} and
AV 11 (m)1 2)oc{cr(m))2
nn \J" r ~ n •

It may very frequently happen that different images in the available set of
images contain the object in different position. In this case images should be
aligned before averaging. This conclusion follows in particular, from Eq.
8.4.4 in which when DFT basis is used

(8.4.7)

where, according to the shift theorem for DFT, {k~m,/)} are mutual
displacement between m-th anf I-th image components.
When mutual displacements of images are known, images can be aligned
through their resampling with an appropriate interpolation. Methods of
image resampling and other geometrical transformation are discussed in eh.
9. If displacements {k~m,/)} are known with a sub-pixel accuracy, in the
averaging sampies of images will be interlaced according to the mutual shifts
of the images. Therefore the resulting image will have accordingly higher
sampling rate than that of the original image set and, hence, will have higher
resolving power. This resolution increase is frequently referred to as super-
resolution from multiple images.
Mutual image displacements are a special case of general image
geometrie transformations. Obviously, image alignment before averaging
should allow for all transformations.
When mutual image displacements or corresponding parameters are not
known, they have to determined. Determination of image displacements or
342 Chapter8

other geometrical parameters is an example of a general problem of image

parameter estimation. It is discussed in Ch. 10. In particular, it is shown in
Ch. 10, that, for the model of additive uncorrelated Gaussian noise in image
components, mutual image displacements can be determined from the shifts
of the position of image cross-correlation peaks from their positions in
autocorrelation. Image averaging with such an image correlational
alignment is called correlational averaging or correlational accumulation
([13]). The correlation accumulation has found many practical applications,
especially in high resolution electrocardiography and similar biomedical
signal processing ([14]) and for processing electron micrographs ([15]).
8. SENSOR SIGNAL PERFECTlNG, IMAGE RESTORATION, 343
RECONSTRUCTION AND ENHANCEMENT

8.5 FILTERING IMPULSE NOISE

Image distortions in form of impulse noise usually take place in image

transmission via digital communication channels. Signal transmission errors
in such channels result in losses, with certain prob ability, of individual
image sampIes and in replacing their values by random values. Impulse
noise mayaIso appear as a residual noise after applying to noisy images the
additive or speckle noise filtering. In this case, impulse noise is formed from
large outbursts of additive or speckle noise that the filtering failed to remove.
Impulse noise is described by an ImpN-model (see Sect. 2.8.1):

(8.5.1)

where {b k •/ } are sampIes of a perfect image contaminated by impulse noise,

{a k ,/} are the perfect image sampIes, {e k ,/} are a sampIes of a binary
random sequence that symbolizes the presence (e k,/ = 1) or the absence
(e k •1 = 0) ofimpulse noise in the {k,l)-th sampIe, and {n k ,/} are sampIes
of yet another random sequence that replace signal sampIes in cases when
impulse interference take place.
From this model it follows that filtering impulse noise assurnes two
stages:
detecting signal sampIes that have been lost due to the noise and
estimating values of those lost sampIes.
Impulse noise appears in images as isolated noise sampIes (see, for
instance, Fig. 2.21, a). Hence, the task of detecting impulse noise sampIes
may be treated as a special case of the task of object localization in clutter
images treated in Ch. 11. In this chapter, the design of optimal linear filters
is discussed that maximize their response to the target object with respect to
the standard deviation of the signal for non--target image component. For
detecting impulse noise sampIes, individual pixels replaced by noise are the
localization target. As follows from results obtained in Ch. 11, optimal linear
filters in this case can be approximated by convolution of two Laplacians
(see Sect. 11.6). Detection of impulse noise sampies can be implemented by
comparing of the output of such a filter in every pixel with a certain
threshold. The reliability of the detection can be increased if it is
implemented in an iterative fashion with decreasing the detection threshold
in course of iterations.
As for estimating image sampIes that have been found lost, they can be
replaced by values predicted from those sampIes in their vicinity that are not
344 Chapter8

marked as replaced by noise. The prediction can be implemented in different

ways. The simplest is weighted averaging of these sampIes.
This reasoning lies in the base of the following simple algorithm for
filtering impulse noise:

"(I-I) ifl"(/-I)
ak,l
,,(tl _ , I ak,l "(H)I > De~,
- ak,l t'T''h r (I)
akr - --- { , (8.5.2)
'a(/-I)
k,l , otherwise

where {a~/,~} are filtered image sampIes on t-th iteration, t = 0, ••• , Nil '
{a(O)
k,l = bk,r'} a(/-I)
k,t are "predicted" sampIe values found by averaging of
neighbor pixels:

a"(H)
k,l
-_ -81 [~
L. ~L.a"(I-I) -a"(I-I)]
n=-lm=-] k-n,l-m k,l '
(8.5.3)

and DetThr(t) is a detection threshold that IS changing In course of

iterations as

DetThr(/) = DetThr _(DetThrmax -DetThrm1n ](t_l). (8.5.4)

ma, Nil -1

As a practical recommendation, the number of filtering iterations can be

set from 3 to 5, the upper threshold DetThrmax is takenof the order of
1/3 + 1/4 of the image dynamic range and the lower threshold should be
selected commensurable with the local contrast of image details that have to
be preserved from false detecting as noise. For good quality digital images
with gray values in the range [0 + 255], this threshold can be set as
15+ 20.
In order to avoid iterations, this algorithm can be replaced by a recursive
filtering in a sliding window in which detection and estimation for each pixel
is carried out from the analysis of image sampIes that have been processed
on the previous filter window position. For filtering along image rows
(index I), this filter implementation can be described as:

ak,t = {~k.l' iflak,t -ak,tl> DetThr (8.5.5)

ak,t' otherwise
 8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 345
RECONSTRUCTION AND ENHANCEMENT

where predicted values Qk ,l are computed as in DPCM coding (see Sect.

3.7.2, Eqs. 3.7.12,3.7.13).
Fig. 8-21 iIIustrates filtering impulse noise in an image with probability of
impulse noise 0.3 by the iterative and recursive filters. One can see that the
filtering makes unreadable image to be readable.

~'l~;;~' ..u;d. I.~ ler401l.j.·titf; :.~:~~'<err~ ! ~.~: i~~·~ly. ~:itr

~ .h.J ue &nhXIIZ thc: ~ flff'.d... . ~. ilIu ~' ~ 't,l\C' ('~~ fb1"~4*,.
Irty . iAY«Yt' ~ttY. ~ ~f.l4J11 ~':\a,~e'~ ~~:~ll ~l<'---:!t~...l · .
!'(Cmi'IHIt t~ ~~ nf \j1I1e1o~~ Si iit.l~~:.~~ ~.q.; :iTr ~~~ c.:.
im. (l f (he' l1:tfeSl · mo{~ . S!.t~eC l!1i... . ~~'of ~b. li;iid " mol~e:«: S~a!~ irl ':li-.
~'tu'!!t-Y .Pf~u ~r.~&.K: CJv a o:'~~'~i~~ .' ~"d1~~: ~jy(::
'*' p:od-=.?1 • . i.;l1fPf{~?t, ~t~.rifJl1<tö :it.p~-i~ iJl'J~~-';J; ~~il,..ri;d
r:xr~ ~c.aki ; llo!o': ic,.".~ l.()j'I ~s.f ~,:~,"lAlc;f.· l.~l(. ~~~·.~~.n' l~jj!
H'.~J',~ldci ,io F?o'(\I"r) ;a7a~e1~_ )t~~·~ht;.~l~!:.~~~li ~~~~/o:c
11.,h cotrullfeO b:i noi.•. do. ti."'~"iu \;!{ .c, tf~ hj.6ti:jit '~.fJIIf,:r.\t.iu'1J
~ Ull ,.~ amdQWt trtt 2ffM.~ lJl 'l~ ~'~I>:~~e ~ ~ifit~ tjf i~~'
~ '1:~Ka.I ~s 'ir. I! .. "in;.f t un ~.:.e.~. ~~'~.f'. ~~ "1',."),Iä",.'
~i!m tCl{ i'JenUOUriul/(iotn" Y/lut ~it!unf~r~~i.lfl"'\! ', p "1'M! ~~
11:1.1 (:")1 ir Ix. ~'ult~ W"fI;l(t;f*'1Q:Ö In ..SV, ·st,:1. ~":J;xfmJR~..~~~ '/jl' ~ol:­
"l~ ~ otw~ ue ,,~ kh~ 0).1 ~~ .. ~ ~~. (I '~~.' ~tt i. t,e iirxt~'\"t~' ~
....... .J".... .J i .a. .. : . . . . ~ _ ... ~~\...- ... .:;..;..,..i..1-..!Xi .,...... k. :.,~ tr.- ". -":"' _ . .,.....: ....I. ~ ... \.. ....
b) c)

Figure 8-21. Filtering a printed text image corrupted by impulse noise with iterative and
recursive filters: (a) - noisy image with probability of impulse nooise 0.3; (b) - result of
iterative filtering according to Eq. 8.5.2 after 4 iterations; (c) - result of recursive filtering
according to Eq. 8,5.5
346 Chapter8

The described filters are not the only options for filtering impulse noise.
In general, for filtering impulse noise, non linear filters are required. Some
nonlinear filters for filtering impulse noise are described in Sect. 12.3 .
In conclusion of this section, consider ways of numerical evaluation of the
impulse noise filtering. MSE criterion used in the design of filters for
suppressing additive noise is not applicable for impulse noise. One can
easily see it from comparison of images distorted with additive and impulse
noise with the same MS-error with respect to the noiseless image (Fig. 8-22).

Figure 8-22. Images distorted by additive noise (a) and impulse noise (b) with the same MS-
error with respect to the noiseless image
8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 347
RECONSTRUCTION AND ENHANCEMENT

The evaluation criterion for impulse noise filtering must take into account
the following three factors.
Distortions due to missing noisy sampies and the probability of
missing that determines contribution of these distortions to quality
corruption of the filtered image.
Distortions due to false noise detection and the probability of false
alarms that determines contribution of these distortions to quality
corruption of the filtered image Probability of false noise detection.
Estimation error for those pixels that were correctly detected as
replaced by noise.
Each of these errors can be evaluated in terms of their mean square values.
However, their contribution to the total evaluation is of more involved nature
and depends on application.
348 Chapter 8

8.6 METHODS FOR CORRECTING GRAY SCALE

NONLINEAR DISTORTIONS

In the imaging system model of Fig. 8-1, gray scale distortions are
attributed to the point-wise nonlinear transfonnation unit. Usually in
correcting gray scale distortions, noise described by the stochastic
transfonnations is ignored. In such an assumption, for correcting image
nonlinear transfonnations in imaging systems it is sufficient to apply to
images the corresponding inverse transfonnation. However, in digital
processing one should allow for quantization effects and for the fact that any
nonlinear transfonnation of quantized signals reduces the number of signal
quantization levels. The phenomenon of sticking of quantization levels
together as the result of their nonlinear transfonnation is illustrated in
Fig.8-23.

~l

Quantized I-""
output ,/

values

V
f-"'
Quantized input values

Figure 8-23. Nonlinear signal transfonnation and sticking quantization levels together. Thin
lines on the figure outline 16 unifonn quantization intervals. The curve shows transfer
function of a nonlinear transfonnation. Bold lines indicate 9 quantization levels left after the
nonlinear transfonnation of 16 quantization levels of input values

In principle, for generating correcting nonlinear transfonnations that

can be applied to quantized distorted input signal, methods of signal optimal
scalar quantization described in Sect. 3.6 should be applied. Consider a
diagram shown in Fig. 8-24. Let be known the nonlinear imaging system
transfer function to be corrected as weIl as nonlinear compressing and
expanding functions used for signal non-unifonn compressor-expander
quantization. Project boundaries of quantization intervals of unifonn
quantizer at the output of the compressing transfonnation to its input and
8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 349
RECONSTRUCTION AND ENHANCEMENT

then further to the input of the nonlinear transformation to be corrected , as it

is indicated by arrows in upper left and upper right quarters of the diagram in
Fig. 8.24. This provides boundaries of the quantization intervals of the
perfect signal that has to be restored from the distorted and quantized input
signal. Within these boundaries, find now optimal quantized representative
values of the perfect signal that minimize the quantization error. Finally,
project these values back to the input scale of the expanding transformation
as indicated by arrows in bottom right quarter of the diagram. Obtained
values determine the needed output values of the correcting look up table
(shown by bold circles in the diagram), that have to assigned to quantized
values of the distorted signal.

,-----------~---------
Compressing
transformation Distorted
for nonuniform signal values
quantization
imaging
system
Boundaries
ofuniform Boundaries of
quantization quantization
intervals

quantization

Perfeet
signal
values

Expanding

Figure 8-24. Principle of correcting nonlinear distortions with an account for effects of
quantization
350 Chap/er8

In practical implementations, more simple methods are frequently used.

On of the simple methods to minimize quantization artifacts is
randomization of quantization errors by adding to the quantized signal a
pseudo-random noise uniformly distributed within the corresponding
quantization intervals. Fig. 8-25 illustrates this method on an example of
correcting low image contrast by stretching its dynamic range. False
contours that are visible in the corrected image (Fig. 8-22, b) may be made
invisible (Fig. 8-25, c) by adding to the corrected image pseudo-random
noise uniformly distributed in the stretched quantization intervals.

Contrast corrected image and false contours Destroying false contours by adding
noise to the corrected image

Figure 8-25. Image contrast correction with adding pseudo-random noise for randomization
of false contours
8. SENSOR SIGNAL PERFECTlNG, IMAGE RESTORATION, 351
RECONSTRUCTION AND ENHANCEMENT

In applications, it very frequently happens that the nonlinear signal

distortion transfer function that has to be corrected is not known. In some
cases it can be determined directly from an analysis of the distorted signal. A
typical example to illustrate such an opportunity is correcting nonlinear
distortions of photographie recording of interferograms.
Perfect interferogram signal is a phase modulated sinusoidal signal. If the
interferogram to be corrected contains many periods of the sinusoidal signal,
distribution function of its phase can be regarded to be uniform. One can
analytically compute the distribution function of a sinusoidal signal with
uniformly distributed phase. Therefore, the correcting nonlinear
transformation can be found as a transformation that converts the histogram
of the distorted interferogram into the histogram defined by the distribution
function of the sinusoidal signal with uniformly distributed phase. We will
refer to such a transformation as to histogram matching. Fig. 8-26 illustrates
the histogram matching for correcting non linear distortions of an
interferogram.

~
Imerferogram with distorted gray scale Corrected"'=:''interferogram
,,~/- : ,:;.: ' w / ' - .. .

Graph of the left half of tne cenrral raw ~f

the distorted interferogram

Figure 8-26. An example of correcting unknown nonlinear distortion of an interferogram

352 Chapter8

Yet another example of an application in which the histogram matching

can be used for correcting unknown nonlinear image distortion is creating
image mosaics from sets of images that are mutually overlapping.
Algorithmically, the histogram matching can be implemented through
the histogram equa!ization, a nonlinear transformation that converts images
with whatever histograms into images with a uniform histogram. For the
histogram matching, it is sufficient to find histogram equalization
transformation for image to be transformed and that for the image with the
target histogram. The required histogram matching transformation can then
be obtained by combining histogram equalization transformation for the
image to be transformed and the transformation inverse to the histogram
equalization transformation for the target histogram.
Histogram equalization is also a very useful method for image
enhancement (see Sect. 8.7.2).
8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 353
RECONSTRUCTION AND ENHANCEMENT

8.7 IMAGE RECONSTRUCTION

As it was already mentioned, the term "image reeonstruetion" is
eonventionally used to designate proeessing aimed at image formation in
transform imaging methods sueh as tomography and holography. In
prineiple, image reeonstruetion requires applying to image sensor data
transform inverse to that used to eolleet these data. However, in reality one
eannot implement this ideal seheme. Two major reasons for that are: (i)
artifaets due to diseretization involved in data eolleeting for digital
reeonstruetion in eomputer and (ii) sensor signal distortions due to sensor's
noise, limited resolution, nonlinear distortions and quantization. Therefore,
similarly to the inverse filtering for image restoration (Seet. 8.2.3), data
inverse transformation for image reeonstruetion has to be supplemented with
a proeessing that eliminates these artifaets and eorreeting distortions to a
maximal possible degree.
Correeting proeessing is, generally, nonlinear and spaee variant.
Therefore, it should be applied both before and after the inverse transform
(see Fig. 8-27). Note that the inverse transform ean also be explicitly
ineorporated with eorreeting in the form of trans form "regularization" as it
done in some tomographie reeonstruetion algorithms.

Perfecting Perfecting
sensor's signal: Inverse reconstructed image:
correcting
distortions and
--. transform
f-+ Correcting distortions,
denoising, and
denoising enhancement

Figure 8-27. Principle of image reconstruction by inverse filtering via inverse transform

We will illustrate image reeonstruetion with pre-inverse transform and

post- inverse transform proeessing using as examples image reeonstruetion
from projeetions and from optieal holograms.
Image reeonstruetion by baek projeeting its projeetions is a
straightforward inversion of image projeeting in the tomographie proeess.
Baek projeetion is a very simple operation that sometimes ean be
implemented in pure analog, for instanee optieal, set-ups, with a very high
354 Chapter8

speed. This is however, not the inverse transfonn required for the exact
reconstruction. Images reconstructed in this way are blurred and need
restoration. Left image in the second row of Fig. 8-28 illustrates such an
image restoration from parallel projections (right image in the first row) of a
test image shown in the left image in the first row. Inverse filtering applied
to images reeonstrueted by back projection can substantially improve image
quality and approximate exaet tomographie reconstruetion espeeially if it is
supplemented with a post-proeessing for removing inverse filtering artifacts.
To enable the inverse filtering, one should measure point spread funetion
of the tomographie maehine with back projeetion reeonstruetion. This ean be
done using specially phantom objects such as for instance, a point objeet.
The system PSF uniquely specify tomographie reeonstruction systems as
they can be treated as linear ones.
Let V be a veetor of observed image sampies, V be a result of its
"
reconstruetion in a given imperfect tomographie system, V be the eorreeted
result, Vc be a ealibrating test image for measuring the system point spread
funetion, and Vc is its eopy reconstructed by the system. The inverse
filtering proeessing can be deseribed, in a matrix fonn, as following:

(8.7.1)
where IF is a restoration linear operator obtained as:

(8.7.2)

In order to reduce the computational complexity of the restoration,

all transformations can be carried out in the domain of Discrete
Fourier Transform using Fast Fourier Transform algorithm:

v= IFFT{CD eFFT(V)} ,
IF (8.7.3)

where FFTO and IFFTO are direet and inverse Fast Fourier Transfonns
and CD IF is diserete frequency response of the inverse filter. It ean be found
from FFT(Vc ) and FFT(Vc ) as

(8.7.4)

As the ealibrating test image, it is most natural to ehoose a desired overall

point-spread funetion ofthe system.
8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 355
RECONSTRUCTION AND ENHANCEMENT

If the tomographic system can be modeled as aspace invariant one (such

as, for instance, systems with parallel beams) the above transforrns should be
applied "globally" to the entire volume within applicability of the space
invariance assumption. Otherwise they should be applied to selected regions
of interest small enough to fulfill the requirement for space invariance with
an acceptable approximation.
Right image in the second row in Fig. 8-28 illustrates the result of such a
restoration by the inverse filtering applied to the left image. The filtering was
implemented in the domain of Discrete Fourier Transforrn. As a "desired"
system point spread function, a Gaussian-shaped impulse in coordinates
{k, I} of pixel indices was chosen:

(8.7.5)

with a spread parameter cr c was set to have magnitude of about pixel size.
Frequency response of the "inverse filter" was generated, similarly to
Wiener deblurring filter ofEq. 8.2.21, as

- IF·
IF = z ij
IIFI + E Z • var IFI z)'
(8.7.6)

where IF is defined by Eq. 8.7.4, Vc is the result of back projection

reconstruction of the desired system point spread function V c ' FFT(.) is an
operator of Fast Fourier Transforrn, E Z is a regularizing parameter playing
the role of the parameterSNR in Eq. 8.2.21 and varO is an operator of
computing mean square value.
Right image in the second row in Fig. 8-28 demonstrates the results of
such an inverse filtering of the left image reconstructed by back projection.
The image is intentionally over-contrasted to make reconstruction artifacts
visible. These artifacts can be substantially reduced by non-linear post-
filtering. Left and right images in the bottom row in Fig. 8-28 illustrate this
option. Left image was obtained using iterative sigma-filter (See Sect.
12.3.1). Right image shows difference non-filtered and filtered images.
 356 Chapter8

Back projection reconstruction

Filtered anifaclS: standard deviation 10- 1

Figure 8-28. Illustration of the inverse filtering for image restoration from back projection
reconstruction
8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 357
RECONSTRUCTION AND ENHANCEMENT

Similar "pre-processing inverse transform post-processing"

technology is adopted also for digital reconstruction of holograms (Fig.
8.27).

Oprical hologram
--- ---- ---
--~~~~ - - -- -- --- - -- -- -- -- -

Analog-to-digital
conversion

-Preprocessing of
Computer I
digitized
hologram

,
\
\

,,
I

--
I

--
--
- - - - -----
~ "..~ - -- ,
- ------
,
I
I
I
I
I
,

,
I
Output image I
,,I
I
I
I
,
I
I
\
\

" ,----- ------- -- ----- - --

Figure 8-29. Basic stages ofthe digital reconstruction ofhologram

358 Chapter8

Hologram preprocessing is aimed at correcting signal distortions in

hologram sensor and removing hologram components that may deteriorate
image reconstruction quality. In Sect. 8.2.2, Fig. 8.4, we iIlustrated filtering
periodic interferences that sometimes take place in the process of digitization
of holograms in frame grabbers when analog video cameras are used as
hologram sensors.
A frequent purpose of hologram preprocessing is also eliminating zero
order diffraction spot in the reconstructed image. Zero order diffraction spot
appears in images reconstructed from Fourier and Fresnel holograms due to
the presence in the recorded hologram a constant or slowly changing
component (see Sects. 4.4.2 and 13.3). A simple method to achieve this is
subtraction from the hologram signal its local mean value computed over a
sliding window ([5]).

_ 1 N, N2

r ' ,S =r' ,s - (2N '" "'r (8.7.8)

I + lX2N 2 + 1) L.. L.. ,-k ,s- I'
k= - N , 1=-N 2

where {r"s} is an array of hologram sampies with (r, s) being the sampie
indices. As it is described in Sect. 5.1 .1, local mean in a sliding window can
be very efficiently computed recursively. In this application, window size is
usually 3 x 3 to 7 x 7 pixels. Fig. 8-30, center, shows result of
reconstruction of the hologram shown in Fig. 8-4 after removing zero-order
diffraction spot. Right image in this figure shows result of the hologram
reconstruction after both removing zero-order diffraction spot and periodic
interferences.

Figure 8-30. Reconstruction of hologram without pre-processing (a) and with preprocessing
for removing zero order diffraction spot (b) and for removing, in addition, periodic
interferences (c)
 8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 359
RECONSTRUCTION AND ENHANCEMENT

Fig. 8.31 illustrates elimination of zero order diffraction spot and image
post-processing in digital reconstruction of a hologram recorded in two
exposures by the method ofphase-shifted digital holography.
Normally, phase-shifting method for digital recording optical holograms
assumes taking three or four exposures of the interference pattern of object
and reference beams with phase shifts of the reference beam O,1t /2,1t for
three exposures and O,1t / 2,1t,31t /2 for four exposures ([16]). This gives
the following three or four recorded holograms, correspondingly:

r o = IRefl2 + IObjl2 + 2IRefIObj",;

r" 12 = IRefr + IObjr + 21 Refl Objim ;
r" = IRefl2 + IObjl2 - 2IRefIObj",. (8.7.9)

and

r o = IRefl2 + IObjl2 + 2IRefIObj",;

r" 12 = IRefl Z + IObjl2 + 21 Refl Objim ;
r" = IReflz + IObjl2 - 2IRefIObj,.;
r 3lt 12 = IRefl 2+ IObjl2 - 2I RefI Obj;". , (8.7.10)

where IRefl 2 is intensity distribution in the hologram plane of the reference

beam, IObjl2 is the intensity distribution of the object wave front and
Obj",and Obj;m are orthogonal (real and imaginary) components of the
object wave front. By means of an appropriate combination of the recorded
holograms, one can separate orthogonal components of the hologram for
reconstruction. For instance, for four exposures:

With two exposures that produce two holograms, r o and r"/2' the
reconstruction that uses these two holograms as real and imaginary parts of
the object hologram, gives a reconstructed image hidden behind a heavy zero
order diffraction spot due to hologram components IRefl2 and IObjl2 as it is
 360 Chapter8

seen in Fig. 8.31, a. Hologram preprocessing described by Eq. 8.7.8 allows

to substantially eliminate influence of these components (Fig. 8.31, b).
FiItering speckle noise and remaining after this filtering impulse noise
provides further improvement of quality of the reconstructed image (Figs. 8.
31, c and d) .

Figure 8-31. Removing zero-order Reeonstrueted image denoising; (a) - image reeonstrueted
direetly from two exposure holograms; (b) - image reeonstrueted from two exposure
holograrns after preproeessing aeeording to Eq. 8.7.8; (e) - reeonstrueted image after filtering
speekle noise using loeal adaptive filtering algorithm (Seet. 8.3.2); (d) - result of subsequent
fihering remaining impulse noise (Seet. 8.5)
 8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATJON, 361
RECONSTRUCTJON AND ENHANCEMENT

8.8 IMAGE ENHANCEMENT

8.8.1 Image enhancement as an image processing task.

Classification of image enhancement methods

Image enhancement is a processing aimed at assisting image visual

analysis. In visual image analysis, the user that observes images plays a role
of adecision making machine while images may be regarded as sets of
features that are needed for the decision making and that are represented in a
special way perceivable to the machine.
Human visual system has certain limitations in its capability to perceive
information carried by images. Visual system can not detect brightness
contrasts that are lower then a certain contrast sensitivity threshold. It has
also a limited resolving power. It fails to analyze images corrupted by noise,
especially by impulse noise or highly correlated noise patterns. Prom the
other side, visual system perceives colors, it has 3-D capability through
stereo vision, it is capable of efficient detecting changing images in time.
One can say that visual system has five channe1s per pixel to perceive
information: three for RGB colors and additional two for stereo and for
dynamical vision. Image enhancement is intended to convert images into a
form that makes use of capabilities of human visual system to perceive
information to their highest degree.
Theoretically, image enhancement methods may be regarded as an
extension of image restoration methods. However, in contrast to image
restoration, image enhancement frequently requires intentional distorting
image signal such as exaggerating brightness and color contrasts, deliberate
removing certain details that may hide important objects, converting gray
scale images into color, stereo and movie ones to display three-four-five
component features, etc. In this sense image enhancement is image
preparation or enrichment in the same meaning these words have in mining.
An important peculiarity of image enhancement as image processing is
its interactive nature. The best results in visual image analysis can be
achieved if it is supported by a feedback from the user to the image
processing system.
Image enhancement methods date back to the beginning of the last
century photography and microscopy methods such as unsharp masking,
solarization, phase contrast ([ 17]). Electronic television enabled using more
sophisticated methods for manipulating image contrasts, brightness, color
and geometry. However only digital computers equipped with appropriate
display devices enabled implementation of truly interactive image
362 Chapter8

processing. Flexibility of programmable digital computers make them the

ideal vehic1e for image enhancement.
From the point of view of implementations, image enhancement methods
can be c1assified into four categories: gray scale, or histogram manipulation
methods, image spectra manipulation methods, geometrical transformation
methods and display methods.
Gray sc ale, or histogram manipulation methods implement different
modifications of image display transfer function. We will review them in
Sect. 8.8.2. Image spectra manipulation methods implement modification of
image spectra in different bases, most frequently DFT and DCT ones. They
are reviewed in Sect. 8.8.3. Image geometrical transformation are used in
image enhancement to facilitate representation of 3-D objects in 2-D images.
Methods for image geometrical transformations are discussed in Ch. 9.
Special information display methods for image enhancement are briefly
outlined in Sect. 8.8.4.

8.8.2 Gray level histogram modification methods

One of the features that determine the capability of vision to detect and
analyze objects in images is contrast of objects in their brightness with
respect to their background. While globally image may utilize the entire
dynamic range of the display device, locally, within small windows image
signal has very frequently much lower dynamic range. Small local contrasts
may make it difficult to detect and analyze objects.
The simplest and the most straightforward way for amplification of local
contrasts is stretching local dynamic range of image signal in a sliding
window to the entire dynamic range of the display device. The two
computationally inexpensive methods for stretching local contrasts are max-
min stretching and local dynamic range normalization by local mean and
variance. They are described correspondingly as

" akl -min kl,

a k,1 = ' •
(8.8.1)
maxk,l- mlßk,1

and

"
ak,1 { -
am.. '
= a" , (8.8.2)
amin'
8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 363
RECONSTRUCTION AND ENHANCEMENT

where

a - mean(WJ,Wl)
'ii = g
A

k,l k,l + ii , (8.8.3)

stdev k,l

ak,l and a k •1 are input and output image sampies in the window position
(k,/), maxk,l' mink,l' are maximal and minimal signal values within the
window, a ma• and amin are boundaries ofthe display device dynamic range,

(8.8.4)

is local mean value and

is local standard deviation within the window.

Local dynamic range normalization by local mean and variance is
iIIustrated in Fig. 8.32, upper row. In Sect. 11.4 it is shown that this
operation is also a useful image preprocessing operation that improves the
discrimination capability of the optimal adaptive correlator for target
detection in a complex background.
Alternative methods of amplification of local contrasts are methods of the
direct modification of image histograms: global and loeal histogram
equalization and P-histogram equalization ([4, 18]). Histogram equalization
converts an image {a k./ } with histogram h{q) into an image {ak,/} with a
uniform histogram by means of the foIIowing gray level transformation:

tll:,l

Lh{q)
)11=11....
A
ak,l = qmB>
(
- qmln tI + qmln' (8.8.6)
!h{q)

where qmln and qma. are minimal and maximal quantized values of image
signal (for 256 quantization levels these are 0 and 255).
364 Chapter8

It follows from Eq. 8.8.5 that the slope of the histogram equalization
transfer function ak,1 (a k,/) is proportional to the histogram value:

{).a-kl' och( )
- (8.8.7)
{).a q q=lIk,l'
k,1

This means that histogram equalization results in amplification of contrasts

that, for each gray level, is proportional to its histogram value, or to the gray
level cardinality.(see Sect. 12.1).
P-histogram equalization defined by the equation

(8.8.8)

is a natural generalization of the histogram equalization (KimlYaro) that

makes it more flexible through the use of a user-defined parameter P. If
P = 0, P-histogram equalization is equivalent to the automatie stretching
image dynamic range from (amin, a max ) to (q mln ,q max ). P = 1 corresponds to
histogram equalization. Intermediate values 0:::;; P :::;; 1 result in more soft
contrast enhancement that is frequently better perceived visually.
The described histogram modification techniques can be applied both
globally and locally. In the latter case, the modification is local adaptive.
Histogram equalization and P-histogram equalization are illustrated in
Fig. 8-32 (middle and bottom rows). Fig 8-33 compares global histogram
equalization and P-histogram equalization and their corresponding transfer
nmctions (compare them with the image histogram).
Note that the P-histogram equalization may be regarded as an
implementation of the nonlinear pre-distortion for optimal compander
quantization (Eqs. 3.6.22-23).
 8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 365
RECONSTRUCTION AND ENHANCEMENT

Figure 8-32. Local histogram modification by means of standardization of local mean and
variance (uper row) and by P-histogram equalization (middle and bottom rows)
 366 Chapter8

SO 100 ISO 200 250

Image histogram

250 .-.-. "1 '-'-'-i ---7' i J

----r---lf---t---l---j
i l / ! I I

---lf- --1-- !
200

150 -- --i ---j- ----;

! ! i :
---i-- r---- i ---r--·-1
i ! !

l
! I
100
1 I I I
I j I I!
50 ----r~( r---T--TI
o
50 100 150 200 250

Histogram equalization transfer function

250
i i
200
! I
--r-·-r--r----, ---, 1 ! !

! I
__.---L._I-___ i
L -1.__ I
!
ISO
I
i
Ii I
. I
i
i
!
100 ..-----+--------i--- --4--+---~
! i i I I
50 ---1---- !.---~---t--j
I
OL-~L-~--~--L-~
I
I
i !
i i
SO 100 ISO 200 250

Histogram P-equalization transfer function

The image after P-histogram equalization (p=O.I)

Figure 8-33. Global histogram modification by means of histogram equalization and P-

histogram equalization. Left column (from top to bottom): initial image, result of histogram
eqalization and P-histogram equalization (P=O.I). Right column, from top to bottom:
histogram ofthe initial image and corresponding gray level transformation transfer functions
8. SENSOR SIGNAL PERFECTING, IMAGE RESTORA TION, 367
RECONSTRUCTION AND ENHANCEMENT

8.8.3 Image spectra modification methods

Two image spectra manipulation methods for local contrast enhancement

are the most simple in the implementation: unsharp masking and non linear
spectra coefficients transformations. They are implemented, respectively, in
signal domain and transform domain filtering.
Unsharp masking is defined by the equation:

(8.8.9)

where mean i~··W2) is the image local mean in the window of

(2W, + lX2W2 + l)pixels defined by Eq. 8.8.3 and g is a user defined local
contrast amplification parameter. Window size parameters W, and W2 are
also user defined parameters that are commensurate with the size of the
objects to be enhanced.
Modification of image spectra that results from applying unsharp masking
is determined by the unsharp masking frequency response. As it follows
from Eq. 8.8.9, it is, for image of N, x N 2 sampies, equal to:

TI r.s = 1 + g[1 - sincd(2W. - 1; NI; r )sincd(2W2 - 1; N 2; r)] (8.8.10)

where sincd(·;·;-} is the discrete sinc-function defined by Eq. 4.3 . 18.

Fig. 8.-34 illustrates 1-0 sections of unsharp masking frequency responses
for windows of 3 and 15 pixels.

~", ,,,,,,·,,,,·; · ······ ·· ,,,, .. 1..·..............;· .... ······,,··!········ ......... , ......... ~.,,;
.............." ............ .
2.2
'. f· . . ······· ; ........, j .. . .

·+. . . ·. . . ·j. . . . . . . . :. . . . . . .
1.8 . ,··j················!···············1· .
........... ;W=3T....·......
",· · , ,·· ·· · · · ~· · · · · ··········!·· .. ············f······· ........ !................!..............
1.4
.. ·······~···············i················r············ · .. ,................ !. . . ......
0.2 0.4 0.6 0.8
Nonnalized frequency

Figure 8-34. I-D sections of unsharp masking frequency responses for window sizes 3 and 15
pixels with amplification coefficient g= I

Unsharp masking is a non-adaptive local contrast enhancement procedure.

With unsharp masking, the degree of amplification of image high
368 Chapter8

frequencies is the same for any image. An alternative and adaptive method
for modification of image spectra that results in local contrast enhancement
is P-th law nonlinear modification of absolute values of image spectra
coefficients. It is described by the equation

~ Cl, h
S

Cl" s =~II'""'·s
I,
P
(8.8.9)
JU- ' ,S

where ~ ' .S fand p '.S i are initial and transformed image spectral
coefficients in a selected basis, respectively, and P is a user defined
parameter. As a transform, usually DFI or DeI are used. When 0::; P < 1,
this modification redistributes energy of spectral coefficients in favor of low
energy coefficients. The degree to which individual coefficients are
amplified depends now on the image spectrum.
Such a spectrum modification may be applied globally to the spectrum of
the entire image and locally in a user specified sliding window to spectra of
the window sampIes in each position of the window. In the latter case, the
modification is local adaptive. In Fig. 8-35 one can compare image local
contrast enhancement by means of unsharp masking and global and local
non linear modification of iI!l~ge DeI spectrum.
",.;;in"'.,..",,,....,.,,,.......ItO:"r._-

DCT global speetrum enhaneement, P=O.5 DCT loeal speetrum enhaneement, in the
window 15x 15; P=O.5

Figure 8-35. Unsharp masking and non linear spectrum modifieation for image enhancement
 8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 369
RECONSTRUCTION AND ENHANCEMENT

8.8.4 Using color, stereo and dynamical vision for image

enhancement

The basic principle of using color, stereo and dynamical capabilities of

vision is straightforward. Image is processed to produce several output
images that represent certain several features of the input image. For
instance, this processing may be image sub-band decomposition, image local
histogram P-equalization using several different window sizes, edge
enhancement and extraction with different algorithm parameters (see some
of the algorithms described in eh. 12), object detection, to name a few. The
obtained set of images may then be used to generate animated artificial

Input image

Artifieial stereoscopie images for left and right eye generated using the image loeal mean as a
depth map.

Figure 8-36. Stereo visualization ofan image and its loeal mean. One ean observe 3_d image
using a stereoseope or by looking at the images with squinted eyes. In the latter ease, one
should tl)' to squint eyes until two pairs of images seen with eaeh eye overlap into 3 images.
Then the eentral ofthose three images with be seen as a 3-D image.
370 Chapter8

movie by using generated images as movie frames, or to generate artificial

color images by representing combination of three generated images as red,
green and blue components of the color image that is displayed for visual
analysis, or to generate artificial stereoscopic images for left and right eyes
from couples of the obtained images. In the latter case, one of the processed
images or the initial image is treated as a "reflectance map" of an artificial 3-
D surface and another as its "depth map". Using the reflectance map and the
depth map, one can generate a pair of images for right and left eyes by
introducing to every pixel of the "reflectance map" a horizontal shift
proportional the value of the "depth map" for this pixel. This can be done
using image resampling methods described in eh. 9. Fig. 8-36 illustrates an
example of such an artificial stereo image generated from air photograph of
256x256 pixels treated as a "reflectance map" and an image of its local
means in the window of 45x45 pixels treated as a "depth map".
8. SENSOR SIGNAL PERFECTING, IMAGE RESTORATION, 371
RECONSTRUCTION AND ENHANCEMENT

References
1. N. Wiener, The interpolation, extrapolation and smoothing of stationary times series,
Wiley, New York, 1949
2. A. N. Kolmogorov, Sur l'interpolation de suits stationaires, C. R. Acad. Sci., 208, 2043-
2045
3. D.L. Donoho and I.M. Johnstone, Ideal spatial adaptation by wavelet shrinkage,
Biometrica, 81(3): 425-455, 1994
4. L. Yaroslavsky and M. Eden, Fundamentals ofDigital Optics, Birkhauser, Boston, 1996
5. L. Yaroslavsky, N. Merzlyakov, Methods of Digital Holography, Consultant Bureau,
N.Y., 1980
6. M.D. Levine, Vision in Man and Mashine, , McGraw-HiII, 1985, pp. 110-130
7. L.P. Yaroslavsky, Local Adaptive Filters for Image Restoration and Enhancement, In:
Int. Conf. on Analysis and Optimization 0/ Systems. Images, Wavelets and PDE's, Paris,
June 26-28, 1996, Eds.: M.-O. Berger, R. Deriche, I. Herlin, J. Jaffre, 1.-M. Morel,
Springer Verlag, Lecture Notes in Control and Information Sciences, 219., 1996, pp. 31-
39.
8. L. Yaroslavsky, "Local Adaptive Image Restoration and Enhancement with the Use of
OFT and DCT in a Running Window", Proceedings. Wavelet Applications in Signal and
Image Processing IV, SPIE Proc. Series, v. 2825, pp. 1-13, 6-9 August 1996, Denver,
Colorado.
9. L. Yaroslavsky, Image Restoration, enhancement and target location with local adaptive
filters, in: International Trends in Optics and Photonics, ICOIV, ed. by T. Asakura,
Springer Verlag, 1999, pp. 111-127
10. B.Z.Shaick, L. Ridel, L. Yaroslavsky, A hybrid transform method for image denoising,
EUSIPC02000, Tampere, Finland, Sept. 5-8, 2000
11. ftp:/ift;p.cis.upenn.edu/pub/eero/matlabPyrTools.tar.gz
12. L. P. Yaroslavsky, H. J. Caulfield, Deconvolution ofmultiple images ofthe same object,
Applied Optics, v. 33, No. 11, pp. 2157-2162,1994
13. L. Yaroslavsky, M. Eden, Correlational Accumulation as a Method for Signal
Restoration, Signal Processing, v. 39, pp. 89-106, 1994
14. N. EI Sherif, G. Turitto, eds., High Resolution Electrocardiography, Futura Mount Cisco,
N.Y., 1992
15. B. L. Trus, M. Unser, T. Pan and A.C. Steven, Digital Image Processing of Electron
Micrographs: the PIC system U", Scanning Microscopy, v. 6,1992, pp. 441-451
16. I. Yamaguchi, 1. Kato, S. Ohta, 1. Mizuno, Image formation in phase shifting digital
holography and application to microscopy, Applied Optics, v. 40, No. 34, pp. 6177-618
17. Photography for the Scientist, Ch. E. Engel, ed., Academic Press, London, 1968, pp. 632
18. V. Kim, L.P. Yaroslavsky, Rank Algorithms for Picture Processing, Computer Vision,
Graphics and Image Processing, v. 35, 1986, p. 234-258.
 Chapter 9

IMAGE RESAMPLING AND GEOMETRICAL

TRANSFORMATIONS

9.1 PRINCIPLES OF IMAGE RESAMPLING

Image resampling is required in many image processing applications. It
is a key issue in signal and image differentiating and integrating, image
geometrical transformations and re-scaling, target location and tracking with
sub-pixel accuracy, Radon Transform and tomographie reeonstruetion, 3-D
image volume rendering and volumetrie imaging.
When image geometrieal transformations are required one have to either
map, according to the required transformation, sampies of the output image
onto input image co-ordinate system or map input image sampies onto the
output image coordinate system. Because the latter method does not allow to
keep arrangement of output image sampies on a fixed rectangular grid, the
first method, "backward co-ordinate mapping" one, is adopted. This
principle is illustrated in Fig. 9-1. As one can see from the figure, computing
transformed image sampies requires resampling of the input image in a new
sampling grid defined by the mapping.
Image resampling assumes, that a continuous model of the input image
should be obtained from its available sampies as it is described by
Eq. 3.3.1, (b) and then this model should be re-sampled in the new sampling
raster defined by the backward mapping. In digital processing one can
approximate this model by generating, from available signal sampies, a new
set of sampies that eorrespond to sub-sampling the initial signal in a more
dense grid, the grid density being defined by the required approximation
aceuracy. This can be done in the following way.
374 Chapter 9

Input image Output image

sampling raster sampling raster
, , I

..
-~----~----.,.--
I , I

:.: :"!:.",, ....1=======>- i i :

------~~ '..... .,----~-----,- -
:::':~ " :::
",,' -t-----c.-----o,--
,.... I I I

I I I
\~ ... "". .••. I , I

Figure 9-1. The principle ofimage resampling for geometrical transformations: backward co-
ordinate mapping

Consider first a 1-0 model. Let {at} be a set of available signal sampies.
Generate an L -times extended set {at} of sampies by complementing each
available signal sampie with L of zero sampies:

{ak }=akN8(kL~ k=Lk N +k L; k N =O,1,••• ,N -1; k L =O,1, ... ,L-1;

(9.l.l)

Then reconstruction of the continuous signal from its sampies described by

Eq. 3.3.1, b can be, in digital processing, implemented by interpolating zero
valued sampies in the set {a k } by means of a digital convolution:

- = ~-
ak L...ak-n hnint , (9.l.2)
n

with an interpolation kerne I {h!nt}. In this way a signal is generated with

L times larger number of sampies then that of the original signal. We will
refer to this procedure as to signal zooming, or L -zooming.
Resampling the interpolated zoomed signal to a new sampling grid can, in
its turn, also be implemented by a digital convolution

(9.l.3)

with a sampling kernel {h:mP1 }. Eq. 9.l.3 is a digital counterpart of Eq. 3.3.1,
a) that describes sampling continuous signals.
 9. IMAGE RESAMPLING AND GEOMETRICAL 375
TRANSFORMATIONS

Digital convolution assumes one or another method for signal extension

outside the interval defined by its available sampIes (See 5.1.4). In view of
the advantages of computing digital convolution in DFT domain (see Sect.
5.2) using fast transforms, let's assume cyclic extension of signal {ak } that
implies that the convolutions defined in Eqs. 9.1.2 and 3 are cyclic
convolutions:
_ LN-I
-
a k = L...
~- h int
a(k-n)modLN n ; (9.1.4)
n=O

_ LN-I
- - ~- h smp'
a k - L... a(k-n)modLN " (9.1.5)
n=O

As far as resampling is cyclic convolution based, resampling errors can be

analyzed by the analysis of DFT frequency responses of the interpolation
and sampling kerneIs. From discrete sampling theorem (Sect. 4.3.2) it
follows that, among cyclic convolution interpolation kerneIs, kernel

int • ( ) sin(1tn) (9.1.6)

h" =SIßcd N,N,n = . ( )
NSID 1tnl N

is the only one that assures preservation of initial signal spectral components
and the absence of aliasing terms in spectra of the interpolated signals. We
will call this interpolation method and its versions the discrete sinc-
interpolation.
2-D signal interpolation can be implemented as either separable or
inseparable digital convolution. When carried out in signal domain,
separable digital convolution is computationally much more efficient then
inseparable one (Sect. 5.1.2). If the convolution is implemented in transform
domain, the use of inseparable convolution kerneIs is possible without any
loss of the computational efficiency. Inseparable interpolation kerneIs are
more natural for image processing. In Sect. 9.3 DFT and DCT based discrete
sinc-interpolation algorithrns will be described that, in principle, enable the
implementation of inseparable convolution kerneIs.
376 Chapter 9

9.2 NEAREST NEIGHBOR, LINEAR AND SPLINE

INTERPOLA TION METHODS

The most known discrete signal interpolation methods are nearest

neighbor interpolation, linear (bilinear, for 2-D signals) interpolation and
spline interpolation. For L -zooming signal of N sampies, interpolation
kerne I {h:t(O)} for the nearest neighbor interpolation is defined by the
equation:

) {I, O:s;
h: . t(O)
n = rect(
niL =
n:S; L-l
0, L:s; n:S; NL-l
. (9.2.1)

Module of its frequency response {~;nt(o)I}= IDFTLN {h~nt(o) ~ that

characterizes distortions of the original signal frequency components at the
result of its zooming, is described by the equation

l'lr I- ~ _I sin(ltr
I... int(o)1 OCSlDCd(L-, LN-,r",- (
1
IN)). (9.2.2)
sin nrl LN

Graph of ~!nt(o)l} for L =16, N =128 is shown in Fig. 9-2 a centered

around the symmetry point r=1024. Central lobe of the graph describes
distortions of the original signal frequency components while its side lobes
describe aliasing artifacts. Fig. 9-2, b) illustrates image 8-zooming and
pixelation aliasing artifacts characteristic for the nearest neighbor
interpolation .

..... .~ .. ..... :... ... ........;- .... .

, , ;

0.6 · .... , .. ·· . ;.. ... .:...... ( ..

0.4 .............: ....... : . . . :

0.2

Figure 9-2. Frequency response of the nearest neighbor signal interpolation for 16-zooming
signals of 128 sampIes (a) and an example ofimage 8-zooming
 9. IMAGE RESAMPLING AND GEOMETRICAL 377
TRANSFORMATIONS

The principle of linear and separable linear interpolation along two co-
ordinates (bilinear interpolation) is illustrated in Fig. 9-3 for interpolation of
signal sampies shifted with respect to available sampies by u -th fraction of
the sampling interval (u -th and v th fractions of the 2-D discretization
interval, for bilinear interpolation).

u i a1,2

::::.:::r.·
........
'. a
············l ...
::::::

jf = uva •.• + (1- u)va •. , + u(l- v)a, .• + (1- u)(1- v)a,.,

a) b)
Figure 9-3. The principle of linear (a) and biIinear (b) interpolation. Available signal sampIes
are shown in bold lines. Interpolated sampIes are shown in doted lines

For L -zooming signal of N sampies, interpolation kerne I {h:t(l)} for linear

interpolation is defined by the equation:

n / L, 0 :::;: n :::;: L -1
h~nt(l) = tri(n / L) = { 2 - n / L, L :::;: n :::;: 2L -1. (9.2.3)
0, 2L :::;: n :::;: NL-l

This kerne I is a convolution of the nearest neighbor interpolation kerne I of

Eq. 9.2.1 with itself. Therefore by virtue of the convolution theorem for
DFT, frequency response of linear interpolation is a squared frequency
response ofthe nearest neighbor interpolation:

I... intl oc
l'lr I·SIßCd(L·, LN.,r11JI -_I sin(1tr
2
( / N)) 1
2
(9.2.4)
sin 1tr / LN
378 Chapter 9

It is shown for 16-zooming signal of 128 sampies in Fig. 9-4 (a) centered
around the symmetry point r=I024. An example of image 8-zooming with
bilinear interpolation is shown in Fig. 9-4, b). As one can see on the figures,
aliasing effects are much less pronounced with linear (bilinear) interpolation
while distortions of original signal frequency components within the base
band is more severe.

0.8 ..... ............ ;.... . ............ . ........ ..... .... .. .

0.6 ......... .. ...... :. """" ... " ". .. .. . ........................... .

0.4 ......... ... .. .. . .. ............ .;............... ..

0.2 ......... . ...... ,.... .. .................. .

0 1 . - - -.........- - -- -
o 500 1000 1500
a) b)

Figure 9-4. Frequency response of linear interpolation for 16-zooming signals of 128 sampies
(a) and an example ofimage 8-zooming with bi linear interpolation

Nearest neighbor and linear interpolations are zero order and first order
special cases of spline (B-spline) interpolation ([1]). In the nearest neighbor
interpolation, one sampie of the original signal is involved in calculation of
each sampie of the zoomed signal. In the linear interpolation, two sampies of
the original signal are involved. In R -th order spline interpolation, R + 1
sampies are involved. Interpolation kerne I in R -th order B-spline is defined
as ([1])

h'pi(R)(X )_~(-l)'(R+l)
- ~ ( "
[O(R+l + x _ t )R]
max, (9.2.5)
t =O R + 1 - t ,.t! 2

where x is a continuous variable measured in the units of the discretization

interval. For signal L-zooming, x = niL. A cubic B-spline is often used in
practice:

O~lxl< 1
l~lxl< 2 (9.2.6)
2~lxl
 9. IMAGE RESAMPLING AND GEOMETRICAL 379
TRANSFORMATIONS

A natural requirement to the interpolation kerneIs h int (x) as functions of a

continuous variable x is that they should be equal to one for x =0 and to
zero for other integer x . Splines of the order R > 1 do not benefit from this
property. To be true interpolation kerneIs, they should be appropriately
modified to meet this requirement. The modified splines are called cardinal
splines ([1]).
Fig. 9-5 shows graph of the cardinal cubic spline interpolation kerne I and
its frequency response for 16-zooming signals of 128 sampIes. One can
notice much lower then for the above methods level of side lobes and closer
approximation of the ideal low pass filter that ensures lower level of aliasing
artifacts and distortions of signal frequency components.

0.9
0.8 ...... . , ..........;

0.7
· ··,·~· · ··· · · .. r"'·····r········-r·········
..... .. .... 'f ....... ,. : ....,... ,!, .. ..... .
0.8 ~.,

0.5

······r········r········r········T
·i· ·· · ··· ··t···· ···· ·~····· ·,·+ , · ,··
. . .

0.1 ..... ........ .. ...... ...... .. ..

Figure 9-5. Cardinal cubic spline interpolation kerne I (a) and frequency response (b) for 16-
zooming signals of 128 sampies
380 Chapter 9

9.3 ALGORITHMS OF DISCRETE SINC-

INTERPOLATION

9.3.1 Diserete sine-interpolation by zero padding signal DFT

spectrum

Zero padding discrete sinc-interpolation method follows

straightforwardly from the discrete sampling theorem discussed in Sect.
4.3.2. For a signal of N sampies, it enables generating L equidistant
intermediate signal sampies per each initial one by padding signal OFT
spectrum with, depending on whether N is odd or even number, (L -1)N
or (L -1)N ± 1 zeros.
For odd N , the algorithm is described as (see Eq. 4.3.49):

a =IDFT
k LN {[1- reet r - (N + 1)/2]. DFTN {a J } =
LN-N-1
1 N-I
JL ~an sined(N;N;(k 1L-n)) , (9.3.1)

where r = 0,1, ..., LN - 1 is index of OFT signal spectrum, IDFTLN { } is

operator of inverse Oiscrete Fourier Transform of the size LN and
DFTN { } is operator of OFT of size N and dot • signifies element-wise
multiplication of the arrays.
For even N, zero padding should be carried out in either of two ways
(see Eqs. 4.3.54 and 55):

a = IDFT
k LN {[1- reet ~~~ ~]. DFTN{a k }} =
1 N-I
JL ~ a" sined(N -1;N;(k 1L-n)) , (9.3.2)

or

a = IDFT
k LN {[1- reet r-NI2-1] . DFTN {}}
ak =
LN -N-2
1 N-I
JL ~ an sincd(N -1;N;(k 1L-n)). (9.3.3)
 9. IMAGE RESAMPLING AND GEOMETRICAL 381
TRANSFORMATIONS

Fig. 9-6 illustrates zero padding for an even N .Upper plot on the figure
shows OFT spectrum of a signal of 8 sampies to be 4-zoomed. The left plot
in the second row is a plot of the zero padded spectrum with discarding
central sampie aN / 2 of the initial spectrum. To the right from it, the
corresponding interpolation kernelsincd(N -1;N; x) is shown. The third
row illustrates zero padding with replication of the initial spectrum central
sample a N/ 2 and the corresponding interpolation kernel
sincd(N + 1;N;x) .

5 __------------------~
~L.dIJ
1 2 3 4
a)
5 6 7 e

0.8 •.• .. •. . . . .......

· . . ..
4
· .
. • • . • • • . . • . : • • . • • •.
,
3 & · .
• • • • • • • • • • • • •
0 .6
0.4
. ..•.....
...•...•. . •.• • ••..•
2
0 .2 .•.••.••• ·· .........
. . .
0 .........,..............
o - 0 .2 "----,~0-:'0-~::-L-~~--4~0-:'0--=5-:".00
10 20 30
b)
5 __----~----~----~~ 1 ...

...·, .. ......
.. . .
.
4

3 - . .·. . . . . . . . . . 0 .5 ..... ... .

·
. ... . ..
. .. . .
2 .. .' . ... . .' ... .

o
10 20 30 100 200 300 400 500
~ ~
5 ,....----------r----~----_.,

~~ : j ~ ~ ~ ~~I ,~lli=:
10
f)
20 30 100 200
g)
300 400
-I
500

Figure 9-6. Zero padding signal spectrum and corresponding interpolation kerneIs

Oistorting N / 2 -th signal spectrum sampie by discarding or replicating it

in the above methods of zero padding can be reduced if this spectrum
382 Chapter 9

component is halved and repeated twice in the spectrum zero padding as it is

illustrated in in Fig. 9-6 e). Obviously, such method corresponds to
interpolation

- 1
a = I r La" sincd[± 1; N; (kIL-n)] ,
N-I
k (9.3.4)
vL "1=0

with interpolation kernel

sincd(± 1;N;x)= [sincd(N + 1;N;x)+ sincd(N-1;N;x)]/2. (9.3.5)

It is illustrated in Fig.9-6 on bottom right plot.

As one can see, this interpolation kernel converges to zero substantially
faster then kerneis sincd(N -1;N; x) and sincd(N + 1;N; x). Therefore, it
is less liable to boundary effects. This why such type of the discrete sinc-
interpolation is recommended as a practical solution for signals with even
number of sampies.
An important issue in practical applications of discrete signal resampling
is its computational complexity. Signal zooming by the zero padding method
can be implemented with the use of FFT algorithms for computing direct and
inverse Discrete Fourier Transforms. In this case, signal L -zooming
requires N log N complex operations for direct DFT and NL log NL
operations for inverse DFT, or totally logNL + (logN)1 L operations per
output signal sampie. In principle, zero padding signal zooming requires,
also an additional intermediate buffer for (L -1)N zero sampies. Note,
however, that computational complexity of the zero padding zooming can be
greatly reduced and the need in the intermediate buffer can be avoided with
the use ofpruned FFT algorithms for inverse DFT.

9.3.2 DFT based discrete sinc-interpolation algorithm for signal

arbitrary translation

Zero padding method of the discrete sinc-interpolation has several

drawbacks. It is inflexible because it allows only integer zooming factors.
Practically, zooming factor often should be chosen to be integer power of
two to allow using radix 2 FFT algorithms that are most wide spread. It is
also very inefficient when only signal translations are required as, for
instance, in image rotation algorithm (see Sect. 3.4.1). Unless pruned FFT
algorithms are used that are very rarely available in standard software
packages, it involves excessive computations owing to the need to apply an
IFFT to the extended zero padded signal spectrum.
 9. IMAGE RESAMPLING AND GEOMETRICAL 383
TRANSFORMATIONS

Much more efficient way to implement the discrete sinc-interpolation

follows from the general invertibility property of shifted DFT described by
Eq. 4.3.17 ([2]). It follows from this equation that one can generate an
arbitrarily shifted discrete sinc-interpolated copy of a signal if signal is
reconstructed from its DFT spectrum by ISDFT(p,O). According to this
equation, for signal {an} of N sampies, its discrete sinc-interpolated copy
~~p)} translated with respect to the original signal by p units of the
discretization interval can be obtained as

~~p)}= {~ak sincd[K;N;k-n- p]} =

{ISDFrr·'l{DFr +. ex{- K; 1k)}}}

in x

{exp(in K; 1 (n + p))} =
{IDFrr'l{DFTN{a. ex{-in K; 1k)} {ex{-iln ~ )}}}x
{eXP(in K ;l(n+ p ))} (9.3.6)

We will refer to this signal transformation as to signal p-shift. Flow diagram

of this algorithm for signal p-shift is shown in Fig. 9-7.

~I...F_F_T_~",,-IF_F_T_~
exp(- in K;- 1 k )

Figure 9-7. Flow diagram ofan algorithm for signalp-shift

When K = N , the algorithm implements exact reversible discrete sinc-

interpolation. For the implementation of the interpolation described by Eq.
384 Chapter 9

9.3.4 with sincd{± l;N; x) as the interpolation kernei, signal spectrum

sampie with index NI 2 should be halved before computing an IFFT. As it
was indicated in Sect. 9.3.1, this interpolation is less liable to boundary
effects.
The described algorithm involves input signal modulation and output
signal demodulation by exponential functions exp[- in (K -1}k 1N] and
exp[- in (K -1 Xn + p)1 N]. One can eliminate additional computations
needed for the modulation and demodulation, which are especially excessive
when signal sampies are real numbers. In this case the scheme converts into
the convolution in the DFT domain (See Eq. 5.2.2) with exponential shift
factor 11, = exp[- i2npr 1 N] playing the role of the convolution kerne I
spectrum. To make the convolution kerne I real, one should modify this
factor to secure its symmetry 11!P) = ~~!J dictated by Eq. 4.3.41 for DFT
spectra of signals with real sampies. In this way we arrive at the following
algorithm:

where, for odd N, K = N ,

(p) _ 1
{exp(- i2npr / N r = O,l,...,(N -1)/2 -1 .
(9.3.8)
11, - '- N(p)_, )' ,
\11 r=(N +) 1 12 + 1,..., N - 1 ,

and, for even N, there are the same three options as for the zero padding
discrete sinc-interpolation method:
option K = N -1 ,

exp(- i2npr 1 N 1r = 0,1,..., N 12 -1

{
11!P) = 0, r = N 12 oc
(11~!J, r=NI2+1, ..•,N-l
DFT{sincd[N-l;N;(k - p)]); (9.3.9)

option K = N + 1 ,
 9. IMAGE RESAMPLING AND GEOMETRICAL 385
TRANSFORMATIONS

exp(- i21tpr 1N ~ r = 0,1, •••, N 12 -1

{
TJ~p)= 2cos(21tprIN~ r=N12 oe
~~lJ, r=NI2+1,••• ,N-1
DFT{sincd[N + l;N; (k - p )]}; (9.3.10)

and option K = ±l ,

exP(- i21tpr 1N ~ r = 0,1,•••, N 12 -1

{
TJ~p)= cos(21tprIN~ r=N12 oe
~~lJ, r=NI2+1,••• ,N-1
DFT{sincd[± l;N; (k - p)R. (9.3.11)

Flow diagram ofthe algorithm for even N and the most recommended option
K = ±1 is shown in Fig. 9-8.

Iexp(i21tpr 1N) I
r = 0,1,•••, N 12 - 1 a. ,(p)

u,
FFT ~--~-----H
r=N12

Figure 9-8. DFT based discrete sinc-interpolation algorithm for p-shifting signals of even
number N of sampIes (option K =±1 ).

We will refer to this algorithm as to DFT based discrete sinc-

interpolation algorithm. Computational complexity of the algorithm, with
the use a FFT for direct and inverse DFT, is O(2NlogN) complex
operations or O(21ogN) per signal sampIe which is substantially less that
the complexity of the zero-padding algorithm. The algorithm is also
completely free of other above-mentioned drawbacks of the zero-padding
algorithm.
 386 Chap/er 9

9.3.3 DCT based discrete sinc-interpolation algorithm for signal

arbitrary translation

DFT based discrete sinc-interpolation described is liable to boundary

effects caused by its implementation as a cyclic convolution. As it was
described in Sect. 5.2.2, boundary effects can be substantially reduced by the
implementation of digital convolution in the DCT domain. Interpolation
equation for signal {at} of N sampies can be in this case obtained from
Eqs. 5.2.10 and 5.2.12 as:

{ai )}= {~a" sincd[K;N;k-n- p]} = ISDFT1/1,0 {DCT{a ~,(p)}}=

p
t }·

_1_{a DCTn
.J2N 0 '10
+2~aDCTn"cos[1t
f:t' '1,
(k+1I2),]_
N

2~a:CTYJ:m sin[1t (k +;' 2),]} , (9.3.12)

where
(9.3.13)

and
n =O, .••,[N 12]-1
n= [N /21 ••• ,[N /2]+N-1
n= [N 12]+N,••• ,2N-1
(9.3.15)
with K = N for odd N and K = ±1 for even N .
By virtue ofthe relationships (9.3.9-11), coefficients 1'h, (p) with even
indices can be found direcdy as

1'hr (p) = ]v exp( i21t ~). (9.3.16)

Therefore one needs to additionally compute from Eq. 9.3.14 only terms
{r, 2r+l (p)} with odd indices. Flow diagram of this algorithm for generating
p-shifted sinc-interpolated copy of signal is shown in Fig. 9.9
 9. IMAGE RESAMPLING AND GEOMETRICAL 387
TRANSFORMATIONS

................................. ~ ..................... .
1.............................
Initial signal la k ) 1. p-+;
{!" (P)}
.........................,...
"'-

: p-shifted sinc-interpolated signal

l.... ... ................
~ "
~k+P} . .......................... ..
-

Figure 9-9. Flow diagram of a DCT based algorithm for p-shift signal of N sampies

We will refer to this algorithm as to Der based discrete sinc-interpolation

([3]).
Figs. 9-10 and II illustrate signal and image zooming with OFT based and
OCT based diserete sine-interpolation. Zooming was earried out by
repetitive applieation of signal shifts as it is deseribed in Seet. 9.4.2. One ean
notiee heavy boundary effeets in form of oseillations that appear when OFT
based interpolation is used and that praetieally eompletely disappear when
the OCT based interpolation is used instead.

J.

a) b)

Figure 9-10. Signal zooming with DFT based (a) and DCT based (b) discrete sinc-
interpolations
388 Chapter 9

Figure 9-11. Zooming image fragment with DFT based (upper right) and DCT based (bottom
right) discrete sinc-interpolations

The DCT based diserete sine-interpolation is eomputationally effieient if

regular (equiduistant) signal resamplimg is required. In this ease it enables
eomputing p-shifted sine-interpolated eopy of the signal of N sampIes with
the eomplexity of O(N log N) operations or O(log N) operations per
signal sampIe. Above-deseribed DFT based diserete sine-interpolation
algorithm has the same, by the order of magnitude, eomplexity. However
fast algorithms for DCT, mCT and IDST require less eomputations than
FFT and IFFT involved in the DFT domain algorithm (see Seet. 6.4). This,
along with less liability to boundary effeets makes the DCT based algorithm
preferable in applieations. In eonclusion note that both DFT and DCT based
algorithms ean implement, with the same eomputational eomplexity, not
only diesrete sine-interpolation, but interpolation with arbitrary interpolation
kernel as weIl. One only needs to speeify the kernel by its DFT speetrum.

9.3.4 Sliding window adaptive discrete sinc-interpolation algorithms

When, as it frequently happens in signal and image resampling tasks,

required signal sampIe shifts are different for different sampIes, above
deseribed diserete sine-interpolation algorithms have no efficient
eomputational implementation. However, in such applieations they ean be
implemented in sliding window ([4]). In proeessing signal in sliding
window, only those shifted and interpolated signal sampIes that eorrespond
to the window eentral sampIe have to be eomputed in eaeh window position
 9. IMAGE RESAMPLING AND GEOMETRICAL 389
TRANSFORMATIONS

from signal sampIes within the window. Interpolation funetion in this ease is
a windowed diserete sine-funetion whose extent is equal to the window size
rather that to the whole signal size required for the perfeet diserete sine-
interpolation. Fig. 9.12 illustrates frequeney responses and PSF of the
eorresponding interpolating low pass filters of two different window sizes
for signal 3-zooming. As one ean see from the figure, interpolation filter
frequeney responses deviate from a reetangle funetion, a frequeney response
ofthe ideallow pass filter.

Windowll

.l

----- - -- ~I~ ----- ~ -----

----- -- --
:
-----~-----
- - ~-- - - -.j--- I· -----
---4----- -1 ---...
--1----- 1----
--- -- rr-----;--
----- ;-- .----~--
. ---I---

---•• - ~~I- ~~~·;~t-~::: !!-l-!nnlm--,~M·m~~t~-n: !

- ;

Figure 9-12. Windowed discrete sinc-functions for windows of 11 and 15 sampies and the
corresponding interpolator frequency responses for signal 3-zooming

Sueh an implementation of the diserete sine-interpolation ean be regarded

as a variety of direet eonvolution interpolation methods. In terms of the
interpolation aeeuraey it has no speeial advantages over other direet
eonvolution interpolation methods sueh as spline oriented ones. However it
offers features that are not available with other methods. These are: (i) signal
resampling with arbitrary shifts and simultaneous signal restoration and
enhaneement and (ii) loeal adaptive interpolation with "super resolution".
For signal resampling with simultaneous restorationlenhaneement, the
sliding window diserete sine-interpolation should be eombined with loeal
adaptive filtering. As it was deseribed in Seet. 8.6, loeal adaptive filters, in
eaeh position k of the window of W sampIes (usually an odd number),
compute transform coefficients {ß r = T {a J}
of the signal {a J
in the
390 Chapter 9

window ( n, r = 1,2,•.•, W) and nonlinearly modify them to obtain

eoefficients {ci, (a, )}. These eoeffieients are then used to generate an
estimate Qk of the window eentral pixel by inverse transform Tk- t { }

eomputed for the window eentral pixel as

(9.3.17)

Sueh a filtering ean be implemented in the domain of any transform though

the DCT has proved to be one of the most efficient. Therefore one ean, in a
straightforward way, eombine the sliding window DCT domain diserete
sine-interpolation signal resampling (Eq. 9.3.12) and filtering for signal
restoration and enhancement described in Sect. 8.6:

{ai}= ISDFT ,o fi: (a: )'11, (p)} .

l12
CT CT (9.3.18)

Fig. 9-13 shows flow diagram of such a combined algorithm for signal
restorationlenhancement and fraetional p-shift. It is assumed in the diagram
that signal p-shift is implemented according to the flow diagram of Fig. 9-9.

DCTin Spe<:trum Spettnlm fuverse

sliding modiliUtiOn
window
modjficatioD
for p-shift
r+ for ~
DCTIDST

.
~t.~1
~elrt

f
Input signal Output signal

Figure 9-13. Flow diagram of signal resampling combined with denoising/enhancement

Fig. 9-14 illustrates application of the combined filtering/interpolation for

image irregular-to regular resampling combined with denoising. In this
example, left image is distorted by known displaeements of pixels with
respeet to regular equidistant positions and by additive noise. In the right
image, these displacements are compensated and noise is substantially
reduced with the above-described sliding window algorithm.
One can further extend the applicability of this method to make
interpolation kernel transform coefficients {t" (p)} in Eq. (9.3.18) to be
adaptive to signal local features that exhibit themselves in signalloeal DCT
speetra ([4]):
 9. IMAGE RESAMPLING AND GEOMETRICAL 391
TRANSFORMA TlONS

(9.3.19)

The adaptivity may be desired in sueh applieations as, for instanee,

resampling images that eontain gray tone images in a mixture with graphieal
data. While the diserete sine-interpolation is eompletely perfeet for gray tone
images, it may produee undesirable oseillating artifaets in graphies.

a) b)

Figure 9-14. An example of image denoising and resampling from irregular to regular
sampling grid: a)- noisy image sampled in a irregular sampling grid; b)- Denoised and
resampled (rectified) image

Input
signal Sliding Discrete DeT based Output
window sinc-interpolator signal
DCT

Analysi.s of local Mber

spectrum

Figure 9-15. Flow diagram oflocal adaptive resampling

The prineiple of loeal adaptive interpolation is sehematically presented on

Fig. 9-15. It assumes that modifieation of signalloeal DCT speetra for signal
resampling and restoration in the above-deseribed algorithm is supplemented
with the speetrum analysis for generating a eontrol signal. This signal is used
392 Chapter 9

to seleet, in eaeh sliding window position, the diserete sine-interpolation or

another interpolation method sueh as, for instanee, nearest neighbor one.
Fig. 9-16 illustrates non adaptive and adaptive sliding window sine-
interpolation on an example of a shift, by an interval equal 16.54 of
diseretization intervals, of a test signal eomposed of a sinusoidal wave and
reetangle impulses. As one ean see from the figure, non adaptive sine-
interpolated resampling of sueh a signal resuIts in oseillations at the edges of
reetangle impulses. Adaptive resampling implemented in this example
switehes between the sine-interpolation and the nearest neighbor
interpolation whenever energy of high frequeney eomponents of signalloeal
speetrum is higher than a eertain threshold level. As a result, reetangle
impulses are re-sampled with "super-resolution". Fig. 9-17 illustrates, for
eomparison, zooming a test signal by means of the nearest neighbor, linear
and bi-eubic spline interpolations and the above-deseribed adaptive sliding
window DCT sine-interpolation. One ean see from these figures that
interpolation artifaets seen in other interpolation methods are absent when
the adaptive sliding widow interpolation was used

-,
-1 .5 L---50-:'::----:-:I00=------::1~50;::------;:;;200~-----::2!::;;_
MYOCTs,nc-shifted 16.54 pixels
15.---~--~--~---'--~

05
o
~5

-,
-15L---50~---:-:100~---:-:'~50;::------;:;;200
=------::2!:~
AdsIlINlVOCTslnc-sh,ftad 16.54 pixelS wilh lhrashold 0 03
'5

05
o
~5

Figure 9-16. Fractional shift of a signal (upper plot) by non-adaptive (middle plot) and
adaptive (bottom plot) sliding window DCT sinc-interpolations. One can notice disappearance
of oscillations at the edges ofrectangle impulses when interpolation is adaptive
 9. IMAGE RESAMPLING AND GEOMETRICAL 393
TRANSFORMATIONS

.--~==-=~~~~~ ..
SL--:;;;;---=-;;=--=.----=. .,. L..-c=---:-::.o:---:"'=---=,.:---:..=---=..=-'
.--~....;.
u-
_---"~-_...- r _.....

' : ·iL',M
:;w. ',i :
-.;. - - ;: -f·'
:
.. - : .
s .., 1IDD 1MJ 3D :.J
.
ur-~;;.:.:.;~==:r--.., ~oc,r.-..: .,..,.,.,.....1Ih.~1'0i . ~I'

Figure 9-17. Signal zooming with non-adaptive and adaptive diserete sine-interpolation. First
row: a model of a eontinuous signal (left plot) and its sampled version (right plot). Seeond
row: initial signal reeonstrueted by nearest neighbor (left plot) and linear (right plot)
interpolation. Third row: signal reeonstruetion using non-adaptive (left plot) and adaptive
(right plot) diserete sine-interpolation. One ean see that smooth sinusoidal signal and
reetangular impulses are reeonstrueted with mueh less artifaets when adaptive interpolation is
used.

Non adaptive and adaptive sliding window sine-interpolation are also

illustrated and eompared in Fig. 9-18 for rotation of an image that eontains
gray tone and graphie eomponents. One ean clearly see oseillations at sharp
boundaries of blaek and white squares in the result of rotation without
adaptation. Adaptive sine-interpolation does not show these artifaets beeause
of switehing between the sine-interpolation and the nearest neighbor
interpolation at the sharp boundaries.
Computational eomplexity of the sliding window DCT sine-interpolation
evaluated in terms of the number of multiplieation and summation
operations per signal sampie is proportional to the window size beeause of
the availability of reeursive algorithms for eomputing the DCT in sliding
window deseribed in Seet. 5.4.
394 Chapter 9

a)

b) c)

Figure 9-18. Image (upper) rotation with sliding window non-adaptive (bottom left) and
adaptive DCT sinc-interpolations (bottom right).
 9. IMAGE RESAMPLING AND GEOMETRICAL 395
TRANSFORMATIONS

9.4 Application examples

9.4.1 Signal and image resizing and localization with sub-pixel

accuracy

One of the immediate applications of above-described discrete sinc-

interpolation methods for generating p-shifted signals is signal zooming and
resizing. With the use of these algorithms, signal L-zooming can be achieved
as a combination of (L-I) successive signal shifts with steps
p = t / L; t = 1,..., L -1 as it is shown in schematic diagram of Fig. 9-19a.

Input Zoomed
signal spectrum
a) b)

Figure 9-19. Schematic diagrams of signal zooming (a) and spectral analysis with sub-sample
resolution (b) using SDFTs for zooming and subsampling factor 3

Image zooming using this algorithm can be implemented as a separable

zooming along both image coordinates.
A natural application of image zooming is image resizing and arbitrary
geometrical transformations. A zoomed image can be regarded as an
approximation to its continuous prototype, the approximation accuracy being
396 Chapter 9

defined by the zooming faetor. Then one ean re-sample the zoomed image to
the required sampling grid. If the number of pixels of output image is not
less then the number of pixels of the initial image, the simplest nearest
neighbor interpolation ean be used for the resampling. One ean also use the
combination of the diserete sine-interpolated zooming and bilinear
interpolation for the resampling in order to reduce the zoom factor needed
for the given resampling aceuraey. Prineiple of the "zooming-resampling"
method of image geometrieal transformation is illustrated in Fig. 9-20.

Figure 9-20. Principle ofthe "zooming-resampling" method ofimage geometrical

transformations

Image geometrical transformation by resampling zoomed image ean be

very eomputationally efficient when many different geometrieal
transformation are required for the same image as, for instanee, it is
frequently the ease in loealization of target objeet of unknown size or/and
orientation.
When the number of output image pixels is less then that of the initial
image, resampling should be carried out with an appropriate low pass
filtering to avoid aliasing artifacts. The importance of proper low-pass
filtering for image resizing is illustrated in Fig. 9-21. In this example, image
is resized by reducing with reduetion faetor 77/256 and then it was
magnified with magnifieation factor 256/77.
The same idea of using SDFT for signal p-shift with the diserete sine
interpolation ean be employed for objeet loealization with sub-pixel
aeeuraey for signal spectral analysis with sub-sample resolution.
 9. IMAGE RESAMPLING AND GEOMETRICAL 397
TRANSFORMATIONS

For object localization with sub-pixel accuracy, correlation peak (See Ch.
10) should be sub-sampled. In this case, only signal sub-samples in the
vicinity of the highest correlation peak are required. Hence, inverse SDFTs
in the flow diagram of Fig. 9-19, a) can be carried out with pruned
algorithms described in Sect. 6.4.
Schematic diagram of the signal spectral analysis with sub-sample
resolution is show in Fig. 9-18, (b). Fig. 9-21 shows an example of such an
analysis for a sinusoidal signal sin(21tkF / N)( k = O, ...,255;F = 36.7).

Figure 9-21. Image resizing with different resampling filters : (a)- initial image; b) - bilinear
interpolation; c) - bicubic interpolation; d) - nearest neighbor interpolation; e) - sincd-
interpolation in window 11 x 11; t) - sincd-interpolation in window 31 x31

:. .. ••••• ";0 .. . _. :_. ___ ••• ••• •••••• -: •• -_..

+..... ----.+..... ....
.~ ~ ~ --

:I ••••• ~ ~ -.+--.--~--

': :::::t:::: i":::::t:::::t:::::t:::::t:

..• •_._-- ·····+···--i---···i--_ ···+·
+.... ;_ •••• •
•••• -:;. . •• • •••••• -: ••••• , '\, - - - - - -.0:.-

+ ....--····+·····i······{------1··
~

u •.. ..
eo so t~
,-~

b)
.....-~.".--, .-
._+- ~'" ,
..• _............ . -!

. ..
..... --_. .... ':t
a)
.. . .. ... ::: l
'....-,
c)

Figure 9-22. Spectral analysis with sub-sample accuracy. a) - a sinusoidal signal b) - its DFT
spectrum; c) - interpolated spectrum peak
398 Chapter 9

9.4.2 Fast algorithm for image rotation with discrete-sinc-

interpolation

Image rotation by an anglee as a geometrical transformation of

signal co-ordinates can be described as a multiplication of signal coordinates
(x, y) vector by a rotation matrix ROTa :

ROTe [x]y =[c~se

sme
-Sine][x]
cose y
(9.4.1)

In digital processing, physical coordinates (x, y) are represented, given

discretization intervals (Ax,~y), by integer indices ofpixels {k,l}:

[;]=[~~l (9.4.2)

and the rotation matrix is applied to the vector of indices:

ROTe [k]I = [c~se

smS
-sine][k] .
cosS I
(9.4.3)

Eq. 9.4.3 describes the resampling rule that should be applied to input image
to generate its rotated copy. In order to simplify computations, one can
factorize rotation matrix into a product of three matrices each of which
modi fies only one co-ordinate ([5]):

- Sine] = [1 -tan(e 12)1[ 1 ~I; -tan:e 12)]

cosS 0 1 JsinS
(9.4.4)

This implementation of image rotation is known as a three pass rotation

algorithm. It assumes, on each pass, only signal shifts along one of the co-
ordinates: along x on the first pass, along y on the second pass and again
along x on the third pass. Above described DFT or DCT based signal
translation algorithms are ideally suited for the implementation of the shifts.
The work of the algorithm is illustrated in Fig. 9-23 a,b. One can see that
with the use of interpolation algorithms that are implemented as cyclic
convolution, rotation entails characteristic aliasing artifacts at image
boundaries. In order to avoid them, one should inscribe the image into an
array of correspondingly larger size.
 9. IMAGE RESAMPLING AND GEOMETRlCAL 399
TRANSFORMATIONS

First pass

Third pass: rotated Second pass Third pass: rotated

image image
a) b)

Figure 9-23. The principle ofthe three pass image rotation algorithm: a) - rotation without
aliasing; b) - aliasing artifacts owing to the cyclicity ofthe convolution

Initial image 3600 rotated image

Rotation error pectrum of thc error

Figure 9-24. Boundary effects and rotation error in image rotation with discrete sinc-
interpolation
400 Chapter 9

Fig. 9-24 illustrates boundary effects and rotation error on an example of

rotation of a test image (left, top row) by 3600 in 10 x 360 steps. Right image
in the bottom row displays spectrum of the central, not subjected to
boundary effects, part of the difference image between the initial and rotated
images (left, bottom row image). Spectrum amplitude is represented in the
image by image gray scale levels and the spectrum is eentered so that image
borders eorrespond to its highest frequeneies. One can see that the discrete
sine-interpolation seeures preservation of image speetral eomponents within
the base band. All distortions oceurred at the image highest speetral
eomponent that is halved in the interpolation algorithm and outside the
image base-band. The latter errors ean be attributed to boundary effeets in
each of the rotation steps

9.4.3 Signal differentiating and filtered baek projeetion method for

tomographie reeonstruetion

Signal differentiating is an operation that requires measuring

infinitesimal inerements of signals and their arguments. Therefore, in digital
signal proeessing computing signal derivative assumes one of another
method of building continuous models of signals specified by their sampies,
that is one or another method of discrete signal interpolation. Because
differentiating is a linear operation, methods of eomputing signal derivatives
from their sampies can be eonveniently compared in the Fourier transform
domain.
Let Fourier transform spectrum of eontinuous signal a(x) is a(f):

00

a(x)= fa(f)exp(-i21tfx)cif (9.4.5)

-00

Then Fourier speetrum of its derivative will be (- i21tf):x (f) :

ä(x)= ~a(x)=
dx
f[(- i21tf):x(f)]exp(- i21tfx)cif
-00
(9.4.6)

From Fourier transform convolution theorem it follows that signal

differentiating can be regarded as signal linear filtering with filter frequency
response

H diff = -i21tf (9.4.7)

 9. IMAGE RESAMPLING AND GEOMETRICAL 401
TRANSFORMATIONS

Let now signal a(x) be represented by its sampIes {a k }, k = O,I, ...,N -1

and let tx,} be a set of DFT coefficients of discrete signal {a k }:

ak =
1 N-I (
~ La,exp -i2n-
krJ (9.4.8)
vN ,~O N

Then, from the discrete representation of integral Fourier transform

discussed in Sect. 4.3.1 one can conc1ude that discrete representation of
differentiating equation Eq. 9.4.6 is

•
ak
_
-
I N-I
~ L~, a,]exp( k-
diff
-,2n
• r J, (9.4.9)
vN ,~O N

where {a k } are sampIes of derivative of signal a(x) and f,:iff} are defined
as:

-i2nrl N, r = 0,1,... , N 1 2 -1
TJdiff ={ -n 12 r=N12 (9.4.l0,a)
, i2n (N - ;)/ N, r= N / 2 + 1,... , N - 1

for even N and

diff _ { - i2nr 1N, r = O,I, ...,(N -1)/2-1

(9.4.10,b)
TJ, - i2n(N -r)1 N, r = (N + 1)/ 2, ... ,N -1

for odd N. Note that the coefficient ll~%') is chosen in Eq. 9.4.10, a)
according to Eqs. 9.3.4 and 9.3.11.
Eq. (9.4.9) describes ~i~ital filtering in the DFT domain with discrete
frequency response f,:iff J and suggests the following algorithm of
computing derivative of signals specified by their sampIes:

(9.4.11)

Digital filter described by Eq. (9.4.11) is called discrete ramp-filter.

Obviously, such an implementation implicitly assumes the discrete sinc-
interpolation of the signal. Its computational complexity, with the use of a
FFT, is O(logN).
 402 Chapter 9

Alternative methods of digital computing signal derivatives are those

implemented by signal direct discrete convolution in the signal domain:

(9.4.12)

The following simplest differentiating kerneIs of two and four sampIes are
recommended in manuals on numerical methods in mathematics ([6])
hndiff(l} = [-1,1] (94
.. 13 a)

and
h:iff(l} = [-1/12,8/12,-8/12,1/12]. (9.4.14 b)
Both are based on an assumption that, in the vicinity of signal sampIes
signals can be expanded into Taylor series. The first one, obviously, assumes
linear interpolation. One can compare these and above-described ramp-
filtering differentiating method by their frequency responses. Applying N -
point Oiscrete Fourier transform to Eqs. 9.4.13 and 14, obtain:

11,diff(l} = sin(1tr 1 N) exp[.l1t (r + 1/2)] (9.4.13 b)

2
and

11 ,diff(l} = sin(31trl N)+8sin(1trl N) exp [.l1t (r + 1/2)] . (9.4.14 b)

2
Absolute values of these frequency responses are shown in Fig. 9-25
along with the frequency response of OFT based differentiating filter (Eq.
9.4.10). One can see from the figure that they entail certain distortions of
signal spectra on signal high frequencies.

l.2
: :
....... , ... ,',.,......1"..............................:-... -_ .. -............... .
I
i """" .. ''' ..... ,.......... j.............................. t·········· .L. .. ........... .~
0.8
........ ···j······························i················,·," """""1",,,
0.6
0.4
0.2 ··· · ········1·· · ··· ·· ··· ·· ·· ·· ·· ···· ··· ···· · ~· ···· · · · · .....

o -0.4 -0.2 0 0.2 0.4

Nonnalized frequency

Figure 9-25. Frequency responses ofdifferentiating filters described by Eqs. 9.4.13 (curve 1),
Eq. 9.4.14 (curve 2) and Eq. 9.4.10 ("Ramp")
 9. IMAGE RESAMPLING AND GEOMETRICAL 403
TRANSFORMATIONS

Signal differentiating by ramp-filtering is used, in particular, in digital

image reconstruction from its projections by the filtered back projection
method (Sect. 2.5, Eqs. (2.5-6, 7». The discrete Jiltered back projection
image algorithm of image reconstruction that uses the ramp-filter and image
rotation algorithms described in Sect. 9.4.1 is as following

{a ,!} = ~lROTe {{pies )}® [1/}};

k k,1 =O,I,..•,N-l (9.4.15)
.~O

where

(9.4.16)

is a vector-row of sampIes of ramp-filtered image projection {pie,)}

obtained for angle a., s = 0,1,..., Ne -1, tt/}
is a vector-column of N ones
and ® symbolizes matrix Kronecker product.

9.4.4 Signal integrating and reconstructing surfaces from their slope

Digital signal integrating is yet another operation that requires

interpolation of discrete signals. It has numerous applications in optical
metrology. One of the most typical applications is surface reconstruction
from their slope measured by optical sensors (see, for instance, [7]).
Similarly to signal differentiating, signal integrating assumes operating
with infinitesimal signal increments and can also be treated as signal
convolution. In the Fourier transform domain, it is described as

ä{x)= fa{x}ix = 1~{f)exp{- i21tfx}ix =

_",2nf
00

f Hint {f)x (f )exp{- i21tfx'pf . (9.4.17)

where a{x) and a{f) are signal and its Fourier transform spectrum,
respectively, and

(9.4.18)

is the integrating filter frequency response.

404 Chapter 9

In digital processing, integrating filtering described by Eq. 9.4.17 can be

implemented in the DFT domain as:

(9.4.19)

where {a k }, k = 0,1,000,N -1 is input signal to be integrated and

0, r=O
N
r = 1,2000' N /2 - 1 ,
r=N /2

11~-r' r = N /2 + 1,000,N-1 (9.4.20a)

for even N and

0, r =0
11 (int) ={ N .
r --.-,r=1,2,0.0,(N-l)/2'
,2nr

11 ~-r' r =N /2 + 1'0'0' N -1 (9.4.20b)

for odd N. Similarly to digital differentiating, digital signal integrating

according to Eq. 9.4.19 automatically implies signal discrete sinc-
interpolation. Note the coefficient 112~~ is chosen in Eq. 9.4.20, a) according
to Eqs. 9.3.4 and 9.3.11.
A number of other numerical integration methods have been described in
the literature. Most known numerical integration methods are the Newton-
Cotes quadrature rules ([6,8]). The three first rules are the trapezoidal, the
Simpson and the 3/8 Simpson ones. In all the methods, the value of the
integral in the first point is not defined because it affects to the result
constant bias and should be arbitrarily chosen. When it is chosen to be equal
to zero, the trapezoidal rule is:

-(T) _ 0 -(T) _ -(T) 1( ) (9.4.21)

al - , ak - a k _ 1 +2"\ak - 1 +a k •
9. IMAGE RESAMPLING AND GEOMETRICAL 405
TRANSFORMATIONS

In the Simpson rule, the seeond point needs to be evaluated by the

trapezoidal rule, then

-es) _ 0 -es) _ -es) 1, 4 ) (9.4.22)

al - ,ak - ak - 1 + "3\ak - 1 + a k _ 1 + a k •

In the 3/8-Simpson rule, the seeond and the third points need to be
evaluated by the trapezoidal rule and the Simpson rule respeetively. Then

ii(3I8S}
o = 0, ii(3I8S}
k
= ii(3I8S}
k-3
+~
8 (a k-3 + 3ak-l + 3ak-I + a k ). (9.4.23)

In these integration methods, a linear, a quadratie, and a eubic

interpolation, respeetively, are assumed between the sampled slope data. In
the cubic spline interpolation, a eubic polynomial is evaluated between every
eouple of point ([9]), and then, an analytical integration of these polynomials
is performed.
As in the ease of digital differentiating, we will compare the integration
methods in terms of their frequeney responses. Frequeney responses for
trapezoidal, Simpson and 3/8 Simpson's integrators ean be found from Eqs.
9.4.21 - 23 as folIows:

(9.4.24)

-es} _ 1 ( 4 )
a-eS}
k - ak-l -"3 a k - 1 + a k _ 1 + a k (9.4.25)

(9.4.26)

By N point Discrete Fourier Transform ofthese expressions we obtain

a;.(T)(l- exp(i27tr I N))= !cx,[l +exp(i27tr IN)]; (9.4.27)

2

a;.(S)(l- exp(i47tr I N))= !cx,[l + 4exp(i27tr I N)+ exp(i47tr IN)];

3
(9.4.28)
406 Chapter 9

(lr(318S)[I_ exp(i61tr / N)] =

~ur[1 + 3exp(i2nr / N)+3exp(i4nr / N)+ exp(i61tr / N)], (9.4.29)

8

Therefore the frequeney responses of these integrators are, respeetively:

0, r=O,
,,!T) = ~
-(Tr) {
= _ cos(nr / N)
r = 1,..•,N -1 '
(9.4.30)
ur 2i sin{nr / N)'

0, r=O
,,!S) = ~
-(S) {
= _ cos(2nr IN)+ 2 r =1,••. ,N -1'
(9.4.31 )
ur 3isin(2nr/N)'

0, r=O
,,!3S) =~
-(3S) {
= _ cos(37tr I N)+ 3cos(nr IN) r = 1,••. ,N -1 .
Ur isin{37trl N) ,
(9.4.32)
In Fig. 9-27, absolute values ofthe frequeney responses ofthe DFT based
method of integration (Eq. 9.4.20) and of the Newton-eotes roles are
represented with a frequency coordinate normalized to its maximal value.
Beeause the absolute values of frequeney responses are symmetrie, only half
of the eurves are shown.

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

Normalized frequency

Figure 9-26. Frequency responses of four types of integrators

 9. IMAGE RESAMPLING AND GEOMETRICAL 407
TRANSFORMATIONS

From the figure one can see that the integration methods are low pass
filters, and that they are almost identical in low frequencies while in higher
frequencies 3/8-Simpson, Simpson and trapezoidal integration methods tend
to introduce integration errors. The frequency response of the 3/8-Simpson
rule tends to infinity for the 2/3 of the maximum frequency, and the
frequency response of the Simpson rule has almost the same tendency for the
maximum frequency. This means, in particular, that round off computation
errors and noise that might be present in input data will be amplified by
Simpson and 3.8-Simpson in these frequencies.

9.4.5 Polar-to-Cartesian coordinate conversion, interpolation of

DFT spectra ofprojections and the direct Fourier method of
image reconstruction from projections

Polar-to-Cartesian and Cartesian-to-polar coordinate transformation is

often needed in optical metrology, digital holography and image processing
because optical sensors and recording devices are frequently work in polar
coordinate systems while computer algorithms are alm ost always assurne
Cartesian coordinate system. It is sometimes useful also as a pre-processing
step to match image natural geometry to square sampling grid oriented
algorithms. A typical example is processing image slices in tomography
illustrated in Fig. 9-27.

Initial image Same image in apolar eoordinate system

Figure 9-27. An example of Cartesian-to-polar coordinate eonversion of an MRI tomographie

slice
408 Chapter 9

Polar-to-Cartesian co-ordinate conversion is a key step in the direct

Fourier algorithm of image reconstruction from projections described in
Sect. 2.5. The zooming-resampling method of image geometrical
transfonnations is a natural possible candidate for the polar-to-Cartesian
coordinate conversion needed for the algorithm because 1-0 OFT of
projections can be combined with OFT based zooming along the radial
coordinate.
Flow diagram of the image reconstruction algorithm with polar-to-
Cartesian coordinate conversion by zooming-resampling method is shown in
Fig. 9-28 (a). Fig. 9-29, (b) illustrates resampling from polar to Cartesian
coordinate by the nearest neighbor interpolation.

,... ...
1-D OFf OCT based Polar-to- Cartesian
spectra zooming coordinate 2-D
zooming along angle r+ conversion of the lDFT
according to coordinate zoomed array of
flow diagram of 1-D OFT spectra of
Fig. 9-24, (b) projection by

T
2-0 Array of
nearest neighbor of
bilinear
interpolation
~
Reconstructed
image projections image
(in coordinates
radius-angle) a)
• Initial polar grid
o 2-Zoomed polar grid
• Polar grid
X Tar~et Cartesian grid
• ' 10 nodes

b) c)

Figure 9-28. Polar to Cartesian grid conversion

Zooming factor is an important parameter of Polar-to-Cartesian co-

ordinate conversion using the zooming-resampling method. The higher is
zooming factor the better is resampling accuracy and the higher is
computational complexity. One can reduce the computational complexity of
polar-to-Cartesian coordinate conversion according to the flow diagram of
 9. IMAGE RESAMPLING AND GEOMETRICAL 409
TRANSFORMATIONS

Fig. 9-28, a) by choosing variable zooming factor for zooming the array
along the angle coordinate. As it follows from the geometry of polar
coordinate system (Fig. 9-28, c», zooming along the angular coordinate for
smaller radial coordinates can be made with smaller zooming factor without
compromising interpolation accuracy.
This principle of variable zooming is explained in Fig. 9-29 ([10]). Fig.
9-30 iIIustrates an example of image reconstruction with this method.

Initial array
\-51 step: zooming
along radial coordinate
:.-:~: .
~- - '-

,,,, "",
econd "'''
step:
zooming Zoomed array
along angle
coordinate

Figure 9-29. Zooming spectra ofprojections with variable zooming factor

Figure 9-30. Shepp-Logan phantom (a) and a result ofits reconstruction from 180 projections
with the DCT based interpolation and variable zooming factor I to 5 (b)
410 Chapter 9

References
1. P. Thevenaz, T. Blu, M. Unser, Image interpolation and resampling, Handbook of
Medieal Imaging, Proeessing and Analysis, I. N. Bankman, Ed., Aeademie Press, San
Diego CA, USA, pp. 393-420, 2000
2. L.P. Yaroslavsky, Effieient algorithm for diserete sine-interpolation, Applied Opties,
Vol. 36, No.2, 10 January, 1997, p. 460-463
3. L. Yaroslavsky, Fast signal sine-interpolation and its applieations in signal and image
proeessing,IS&T/SPIE's 14th Annual Symposium Eleetronie Imaging 2002, Seienee and
Technology, Conferenee 4667 "Image Proeessing: Algorithms and Systems", San Jose,
CA, 21-23 January 2002. Proeeedings ofSPIE vol. 4667
4. L. Yaroslavsky, Boundary Effeet Free and Adaptive Diserete Signal Sine-interpolation
Aigorithms for Signal and Image Resampling, Appl. Opt., v. 42, No. 20,10 July 2003, p.
4166-4175
5. M. Unser, P. Thevenaz, L. Yaroslavsky, Convolution-based Interpolation for Fast, High-
Quality Rotation of Images, IEEE Trans. on Image Proeessing, Oet. 1995, v. 4, No. 10,
p. 1371-1382
6. lH. Mathews, K.D. Fink, Numerieal methods using MATLAB, Prentiee Hall, Ine, 1999
7. S. C. Iriek, R. Krishna Kaza, and W. R. MeKinney, "Obtaining three-dimensional height
profiles from a two-dimensional slope measuring instrument", Rev. Sei. Instrum. 66
(1995),2108-2111
8. W.H. Press, B.P. Flannery, S.A. Teukolsky, W.T. Vetterling. Numerieal recipes. The art
of seientifie eomputing. Cambridge University Press, Cambridge, 1987
9. C. Elster, I. Weingärtner, "High-aecuraey reeonstruetion of a funetion f(x) when only
df(x)/dx or d2f(x)/dx 2 is known at diserete measurements points", Proe. SPIE, v. 4782
10. L. Yaroslavsky, Y. Chemobrodov, DFT and DCT based Diserete Sine-interpolation
Methods for Direet Fourier Tomographie Reeonstruetion, 3-d Int. Symposium, Image
and Signal Proeessing and Analysis, Sept. 18-20,2003, Rome, ltaly
 Chapter 10
SIGNAL PARAMETER ESTIMATION AND
MEASUREMENT. OBJECT LOCALIZATION

10.1 PROBLEM FORMULATION. OPTIMAL

STATISTICAL ESTIMATES

Measurement of physical parameters of objects is one of the most

fundamental tasks in image processing. It is requireci in many applications.
Typical examples are measuring the number of objects, their orientations,
dimensions and coordinates. A special case of this problem is also object
recognition when it is required to determine object index in the list of
possible objects. Usually measurement devices and algorithrns are designed
for the use for arbitrary images from multitude of images generated by
image or holograrn sensors. Therefore the most appropriate approach to
solving this problem is statistical one.
Let a(x, p) be an object signal that depends on a certain parameter p ,
scalar or vectorial, that uniquely specifies the signal, P91 (p) is a probability
distribution over an ensemble p E 9t of parameter values (probability of
values if they are discrete of probability density if the parameter is a
continuous variable). Let also h", (x) be a realization of signal that is
generated by the sensor in response to an object signal and is available for
measuring the parameter. Realizations {h", (x)} belong to a statistical
ensemble ro E n generated by the ensemble of parameter values and by
random factors such as sensor's noise that specify a relationship between
signals {a(x, p )} and {hOl (x)} . Define parameter estimation as a
transformation that maps signals {hOl (x)} onto aaset of estimates of
parameter values:

p = ESTIM(hOl (x)). (10.1.1)

412 Chapter 10

Statistically best estimation transformation is the transformation that

minimizes deviations, measured in certain statistical deviation measure, of
the parameter estimates from their true values.
To begin with, consider the case of object recognition, when parameter
p takes discrete values from a set of values {p,,}, numbered by an integer
index q. A natural measure of the estimation transformation performance is
in this case probability Perr of misrecognition. For any given realization
b., (x) of the sensor's signal, parameter value p" may be expected with
probability plp" / b", (x)j. This probability is called aposteriori probability
of parameter value p". If, for signal b., (x), the estimation device generates
estimation p = P" ' error occurs with the probability of all other parameter
values which is equal to 1 - plp" / b", (x )J. On average over the ensemble n
of realizations of b", (x), the probability of misrecognitions will then be

where AVnO symbolizes the averaging over the sensor's signal ensemble.
Because all values involved in the averaging are non-negative, the
probability of misrecognitions is minimal when, for all b", (x), parameter
estimate p is selected that has maximal aposteriori probability:

Popt =argmaxP{p,,/b(x)}. (10.1.3)

{Pq}

Such an estimate is called MAP-estimate and the described approach to the

optimization of parameter estimation is called Bayes approach. According
to Bayes rule:

(10.1.4)

where P[b(x)] is probability density of the sensor signal ensemble {b(x)} ,

p{ b(x)/ p,,} is the probability that object signal a(x, p,,) generates sensor
signal b(x) and p(p,,) is apriori probability of the parameter. Therefore
one can find optimal estimate of the parameter as

A P{b(x)/ pJp{pJ (10.1.5)

p MAP = argmax [( )11 '
{Pq} Pb x ~
 10. Signal parameter estimation. Object localization 413

or, as P[b(x)] does not depend on {p q } , as

(10.1.6)

This equation is the basic formula for the design of optimal Bayes parameter
estimating devices. It requires knowledge of the conditional probability
p{ b(x)/ Pq} that depends on the relationship that exists between sensor
signals {b" (x)} and object signals {a{x, pJ} and of apriori probabilities
{p(p q)} of parameter values. The latter may be unknown. Estimates

(10.1.7)

that ignore apriori probabilities (or, which is equivalent, assume that a

priori probability distribution is uniform) are called Maximum Likelihood,
or ML-estimates.
Eqs. 10.1.6 and 7 were obtained in the assumption that parameter to be
estimated is seal ar and takes values from a discrete set of values. It is quite
obvious that they may be in a straightforward way extended to vectorial
parameters in which case probabilities involved in the equations are
corresponding multi-variate probabilities and to parameters with continuum
of values in which case probabilities should be replaced by corresponding
probability densities.
In what follows we apply the described Bayes approach to solving a
particular problem parameter estimation, that of target location, or
measurement of coordinates of a target object in images. However the
obtained results mayaiso contribute to more deep understanding of the
methods of solving other problems of image parameter estimation such as
problems of image segmentation, object recognition, fitting 2-D curves
("snakes"), 3-D geometrical models and many others. All these problems
may be formulated as model fitting that may be reduced to the
estimation of the model parameters.
414 Chapter 10

10.2 LOCALIZATION OF AN OBJECT IN THE

PRESENCE OF ADDITIVE WHITE GAUSSIAN
NOISE

10.2.1 Optimallocalization device. Correlator and matched filter.

{bk} of the observed image

Consider a discrete model in which sampies
signal can be regarded as a sum of sampies {ak(xo'Yo)} of a target object
signal with unknown coordinates {xo,Yo} and sampies {n k } ofthe noise:

(10.2.1)

Assume that noise sampies {n k } are statistically independent on the signal

{ak (xo, Yo)}' are uncorrelated and have a Gaussian probability distribution
with zero mean and variance er ; . This model describes the simplest situation
in which the only disturbance interfering with object localization is the noise
of the signal sensor which can frequently be regarded as additive, Gaussian,
signal-independent and uncorrelated. Typical practical tasks to which such a
model corresponds are, for instance, those of localization of constellations in
stellar navigation and tracing target objects observed on a unform
background.
For the model of Eq.(10.2.l) conditional probability of sensor signal
sampies {bk} for target signallocated in coordinates (x o,Yo) is equal

(10.2.2)

Therefore the optimal object coordinate MAP estimate is as folIows:

(10.2.3)

As sampIes {n k } are assumed to be uncorrelated Gaussian values with

• 2
vanance cr n'
 10. Signal parameter estimation. Object localization 415

where N is the number of signal and noise sampies. Substitute Eq.(10.2.4)

into Eq.(10.2.3) and, taking into ac count that expO is a monotone function,
obtain for MAP-estimate:

X-I X-I
Finally, because ~:::IbkI2 and Llak(xo,Yof do not depend on the
k=O k=O
coordinates (xo,Yo) we have :

(10.2.6)

Correspondingly, ML-estimate oftarget coordinates may be obtained as

(10.2.7)

According to the theory of discrete representation of signal transforms

(Ch.4):

(10.2.8)

where b(x,y) and a(x-xo,y- Yo) are the continuous signals

corresponding to the sampie sets {bk} and {ak(xo,Yo)}, respectively, and
Ax,L\y are discretization intervals corresponding to the chosen sampling
rate for the discrete model of Eq.(1 0.2.1). Then obtain MAP- and ML-
estimations for continuous signals, respectively, as:

{(x"J,)} ~ arx:~j{llb(X'Y )o(x - x"Y - y,}wty + N, In P(x"y,)},

(10.2.9)
416 Chapter 10

(10.2.10)

where

(10.2.11)

is spectral density of the noise.

Thus, the optimal ML-estimator should compute the cross-correlation
function between the object signal a(x - xo,y - Yo) and the observed signal
b(x,y) and take the coordinates of the maximum in the correlation pattern
as the estimate. Schematic diagram of such an estimator is shown in
Fig. 10-1.

--.... Correlator
(matched filter)
...... Device for localization
of signal maximum
.....
Input
image
i
Target object signal

Figure JO-1. Schematic diagram ofthe optimal device for localization of objects in images

The optimal MAP-estimator also consists of a correlator and a decision-

making device locating the maximum in the correlation pattern. The only
difference between ML- and MAP-estimators is that, in the MAP-estimator,
the correlation pattern is biased by the appropriately normalized logarithm of
the object coordinates' apriori probability distribution.
The correlation operation for a mixture of signal and noise with a copy
of the signal is often caIIed matchedJiltering Correspondingly, the filter that
implements this operation is called matched filter. The correlation may be
implemented in the frequency domain by multiplication of the input signal
spectrum by frequency response of the matched filter which" according to
the properties of Fourier transforrn, is a:(fx,fJ, a complex conjugate of
the object signal spectrum. Correlation, or matched filter type of the
localization devices mayaiso be implemented by optical and holographic
means; a fact recognized at a very early stage in the history of holography
([ 1D.
 10. Signal parameter estimation. Object localization 417

10.2.2 Computer and optical implementations of the matched filter

correlator

Two options are available for computer implementation of the matched

filter correlator: digital convolution in the signal domain or in the Fourier
transform domain as it is described in Ch. 5. The latter is usually more
efficient in image processing because using FFT algorithms reduces the
computational complexity of matched filtering to, by the order of magnitude,
O(logN) operations per sampie, where N is size of the image array. Flow
diagram of the matching filtering in the Fourier Transform domain is shown
in Fig. 10-2.

IFFT
Input image Correlation
b(x) image

Target object image a(x)

Figure 10-2. Flow diagram ofthe matched filtering in the Fourier Transform domain

The most straightforward optical implementations of the matched filter

correlator is known as the 4-F correlator . It is aversion of the optical image
processing system with two lenses described in Sect. 2.3. Schematic
diagram of the 4F-correlator is shown in Fig. 10-3, a). Using parabolic
mirror instead of Fourier lenses makes the correlator to be more compact
([2]). In this correlator shown in Fig. 10-3, b), input image plane, Fourier
plane and correlation plane coincide. If a transparency with an input image is
placed in the left focus of the mirror and is illuminated by a parallel beam of
coherent laser light, its Fourier spectrum appears in the same plane around
the mirror optical axis. In this place, a reflective optical spatial modulator
with reflectance proportional to the frequency response of the matched filter
is positioned. It modifies image spectrum correspondingly. Right part of the
mirror performs the second Fourier transform and reconstructs, in the same
focal plane, a correlation image centered at the mirror's right focus.
For both devices, the use of prefabricated spatial matched filters in form
of an optical or computer generated holograms is assumed. For real time
correlation, the Joint Transform Correlator was suggested ([3,4]). It is an
opto-electronic device schematically shown in Fig. 10-4.
418 Chapter 10

Matched filter

ParaDe)
laser ligh;.;;t_-IIf'IIIt- __ F___ _
beam

Input ] ost Fourier Correlatlon

Fourier 2-nd Fourier
image plane
lens plane lens

a)

,. I ,~...........
,.,- I ,.......... I

,.,.
: " ........... I
,-
I " l .I. . . . . ... ...
,. I I ,

I
,. I
I
I
"
" I
: ' f
I

b)

Figure 10-3. 4-F (a) and a parabolic mirror (b) optical correlators

Correlation
Joint spectrum plane plane
I
,, rVeamera
, I
I
ParaDeI I
beam ......... ~ __ L__ __.f__
oflaser
light . beam ..........•.r..r'-_i'_--
oflaser
light ,,
" ,

lost 2-nd
Spatial
Fourier Fourier
light
lens lens
modulator

Figure 10-4. Schematic diagram ofthe Joint Transform Correlator

 10. Signal parameter estimation. Object localization 419

In the Joint Transform Correlator, input images and target object image
are placed aside in the input plane. Squared modulus ofFourier Transform of
the sum of these images, or joint spectrum of the images, is converted by TV
camera into an electrical signal that controls the transparence of the spatial
light modulator. The joint spectrum is described by the equation

b(x,y)+ a(x + x,y) FOllrle,1'rtm./Or1ll >Iß (fx' /y )+a (tx' /y)exp(i21t/xxt =

Iß{tx,fyf +p{tx,fyf + ß{tx,fJI·{tx,fJexp(-i21t/xx)+
ß•{tx' fy}x (fx' /Jexp(i21t/ x) ,x (10.2.12)

where b(x,y) and a(x,y) are input and target object images and x is a the
displacement, along coordinate x, of the target object image with respect
the system optical axis. As one can see from this equation, the joint spectrum
consists of four components. First two components are power spectra of the
input and target object images. The third component is spectrum of their
correlation and the fourth component is complex conjugate to the third one.
The second optical part of the Joint Transform Correlator performs
Fourier Transform of the joint spectrum. As it follows from Eq. 10.2.12,
output image of the Joint Transform Correlator also contains four
components. First two components centered at the optical axis are images of
autocorrelation functions of input and target object images. The third
component is the image of their correlation function. It is reconstructed
displaced along x axis by X. The fourth component is the conjugate
correlation image. It is reconstructed in the position - x symmetrical to that
of the correlation image.

10.2.3 Performance of optimal estimators: normal and anomalous

localization errors

Owing to the noise present in the sensor, any measurement device will
estimate the object coordinates with a certain error. The optimal ML
coordinate estimator of Fig. 10-1 that determines the object coordinates by
the position of signal maximum at the output of the correlator (matched
filter) provides the most accurate coordinate estimates possible.
Tbe estimation accuracy may be generally characterized by the
J
probability density p(1) x ,I> of the estimation errors:

(10.2.13)
420 Chapter 10

For the evaluation of p(E x,E J one should analyze the distribution
density of the position of the highest peak of the signal at the output of the
matched filter within the area of all possible positions of the target object.
This signal is a sum of the target object signal auto-correlation function and
of a correlated Gaussian noise resulted from filtering the input white noise
by the matched filter. This mixture is a non-stationary random process. This
fact complicates the analysis very substantially. An approximate analytical
solution of the problem can be obtained using following reasoning.

a)

b) c)
" .. -. - - - • -J,\ -. - - - • - --.
,. ---: -- 'f-.--\\'.---:
o. - - -' . • . . '" . . .'
----:
. ..

-!
~
': r:"~~,:'~ . ~ ~ ~ ..\ ~. :;,.;~
d) e)

Figure JO-5. Nonnal and anomalous localization errors: a) - target object signal; b), c) - target
object signal hindered by additive noise of different intensity; d) ,e) - two corresponding
outputs ofthe filter matched with the target signal that correspond to nonnal and anomalous
localization errors, respectively.

Consider Fig. 10-5. Graph on Fig. 10-5, a) represents a target signal of a

rectangular shape. Graphs on Fig. 10-5 b) and c) show mixtures of the target
signal and noise with low and high intensity. Corresponding outputs of the
correlation filter matched with the target object are shown in Figs. 10-5 d)
and e). As one can see from these plots, two essentially different types of
possible estimation errors have to be distinguished. One type of the errors
occur when noise intensity is relatively low (Fig. 10-5, d». In such cases the
correlator output maxima lie in the vicinity of their correct values, and,
therefore, localization errors are also relatively smalI. The second type of the
errors are large errors. They take place when correlator output maxima are
found very far away from the actual position of the target object marked by
the position of its auto-correlation function (Fig. 10-5, e». In this case very
large localization errors occur.
10. Signal parameter estimation. Object localization 421

Therefore small1ocalization errors are caused by distortion of the object' s

autocorrelation peak shape by the noise and large errors are a result of
especially large noise bursts occurring outside the area occupied by the
object. These large errors are similar to the so-called "false alarm" errors in
signal detection. Following V.A. Kotelnikov [5] we call the first type of
error normal errors because, as we shall see, their density distribution may
be approximated by Gaussian (normal) distribution. The errors ofthe second
type we call anomalous errors. The normal errors are responsible for the
behavior of the error probability density p(8 x ,8 y) in the vicinity of zero
errors (8 x = 0,8 y = 0). One can say that normal errors characterize the
accuracy of coordinate measurements. Anomalous errors, obviously,
determine the tails of the error probability density p(8 x,8 y). One can say
that the anomalous errors characterize the measurement reliability. Normal
errors are studied in Sects. 10.3. Anomalous errors are discussed in Sect.
10.5.

10.3 PERFORMANCE OF THE OPTIMAL

LOCALIZATION DEVICE

10.3.1 Distribution density and variance of normal errors

In order to determine statistical characteristics of normal errors, let us

consider the correlator's output signal:

ff[a(s -xo,YJ-Yo)+n(S,YJ)]a(S -x,YJ-y}isdYJ =

Ra (x - xo'Y - Yo)+ Rn (x,y), (10.3.1)

where

RJx,y) = ffa(s,YJ) arS - x,YJ - y)ds dYJ (10.3.2)

X

is auto-correlation function ofthe target object signal, and

422 Chapter 10

n{x,y}= Hn{I;,T)}a{1; -x,T) -y}d1; dT) (10.3.3)

x

is a correlated Gaussian random process resulted from filtering input

Gaussian noise by the matched filter. It is fully determined statistically by its
correlation function

Rn = AVJii{xl'yJ ii{x2 ,yJ]=

ffx ffx
AV {n(~pT), )n(~2 ,T)2 )}a(~, - XpT], - yJa(~2 - X 2 ,T]2 - Y2)d~, dT], ~2 dT]2 =
n

x x
Noffa{;, -xl'T), -y,)a{;, -x2,,,, -Y2)~' dT), =
x
No Ra (x2 -XI'Y2 - yJ, (10.3.4)

where AVn U denotes statistical averaging over the noise n(x,y) ensemble.
In the ca se of normal errors, maxima of the signal R(x,y) at the output of
the correlator are located in the close vicinity of the point (xo,Yo) ' the
position of maximum of RJx,y). The following system of equations
determines coordinates the maximum of R(x,y) :

(10.3.4)

Let the solution of this system be:

X= x o +n x'.
{ (10.3.5)
YA=yo +n·
y'

and localization errors n x and n y are small. Then we have:

(10.3.6)
10. Signal parameter estimation. Object localization 423

The first terms in Eqs.(lO.3.6) may be expressed as:

82 82
+nx - 2 Ra(x-xo'y-yo~x=xo +ny--Ra(x-xo'y-yo~x=xo (10.3.7)
8x Y=Yo 8x8y Y=Yo
=Dxnx + Dxyny,

where

(10.3.8)

by definition because ( {xo,Yo}) are the coordinates of the object, and

(10.3.9)

Similarly one can ob ta in :

(10.3.10)

with

(10.3.12)

Denote

(10.3.13)
424 Chapter 10

Then from Eqs.10.3.6 we obtain the equations:

Dxnx + Dxyny =v x;
(10.3.14)
Dxynx + Dyny =v y

from which we find the following relationships for errors n x and n y in X

and y directions :

D D
n = y v - xy v·
x D D _ D 2 x D D _ D 2 Y'
x Y xy x Y xy
(10.3.15)
n = Dx v -
Dxy v.
y D xDY - D xy2 Y D xDY _ D xy2 x

From Eq. 10.3.15 it follows that small errors {nx,n y } in the

determination of coordinates of the object by the ML-estimator have a
Gaussian distribution with zero mean and the following variances :

-2 DDY xy AV ~}v (10.3.16a)

(D x D y _ D xyJ
2 \2 n x Y

IO.3.16b)

(l0.3.16c)
 10. Signal parameter estimation. Object localization 425

Parameters involved in formulas (10.3.16) may be found using the

relationship between signal correlation fimctions and power spectra and
properties ofthe Fourier transform:

y=Yo

00 00 2

=-41t 2 f f/;~(Jx,/y~ d/xd/y =-41t 2 j;Ea, (10.3.17)

where a(Jx,/J is Fourier spectrum ofthe target object signal:

(10.3.18)

E a is its energy:

(10.3.19)

and 1: is the inertia moment of its power spectrum along the axis Ix

II/x ~ (Jx, /y rd/xd/y

2

IIr (Jx, /y rd/xd/y

12=~-00~-00~____________
(10.3.20)
x

Similarly,

(10.3.21)

and
426 Chapter 10

(10.3.22)

where

j jl:~(I.,I,rdl"dl,
1 =....:-"""-'-:,""=-""-------
JJ~ (Ix, I, rdlßl,
y
1 (10.3.23)

-00-00

and

r
j j 1,,1, ~ (Ix' I, dlßly
r
1- 1
Jxy
= ....:-""::....-"""'---------- (10.3.24)
J
0000

J~ (I", I y dlßl,

Second moments AVn(~,,11), AVn(~,11) and AVn~"vy)ofthe noise

component in the formulas (10.3.16) mayaiso be found in the spectral
domain . For instance, for v x we have:

(10.3.25)

Hence, power spectrum of v x is equal to 4n 11: times power spectrum

of R,,(x,y) which, by virtue of Eq.(10.3.3), is equal to No Ia. (Ix' I, f.
Therefore,

-00-00

In a similar way one can obtain :

(10.3.27)
 10. Signal parameter estimation. Object localization 427

and

-00-00

After substituting these parameters into Eqs. (10.3.16) and some

obvious algebraic transformations we finally arrive at the following
relationships for variances of normal errors in object coordinate estimation
in the presence of additive white Gaussian noise for the optimal ML-
estimator:

(l0.3.29a)

(l0.3.29b)

(l0.3.29c)

Eqs. (10.3.29) demonstrate that variances ofnormallocalization errors are

fully determined by the signal-to-noise ratio No I E. and the inertia
moments (10.3.20-24) of power spectrum of the target object signal. These
are the only characteristics of the object shape that affect its potential
localization accuracy.
Power spectra r{tx,fyf of real valued signals feature the property of
central symmetry:

(10.3.30)

If the object signal power spectrum is symmetrical with respect to the

coordinate axes as well:

(10.3.31)
428 Chapter 10

Eqs.(lO.3.29) take the following simpler form:

(10.3.32a)

(10.3.32b)

er xy2 = o. (10.3.32c)

The last equation shows that if the signal power spectrum is axes-
symmetrical, normal localization errors along coordinates x and y are
uncorrelated. This situation takes place if the object spec!rum a. (Ix' 1,), or,
which is the same, the object signal a(xJ') is a separable function of the
coordinates. Equations (10.3.32) also coincide with the very well known
relationship for one-dimensional signals ([5,6]).
It is sometimes more convenient to express variances of normal
localization errors in terms of the noise variance er ;"t,n at the output of the
matched filter rather than in terms of the input noise spectral density No.
Because this variance is equal to

(10.3.33)

Eqs.(10.3.29) may be rewritten as

(l0.3.34a)

(10.3.34b)
 10. Signal parameter estimation. Object localization 429

(l0.3.34c)

In digital processing, it is natural to represent nonnal error variances in

units of signal discretization intervals Ax,Ay. Denote

(l0.3.35a)

(10.3.35b)

(10.3.35b)

Then, from Eqs. (10.3.29) obtain:

(10.3.36a)

(10.3.36b)

(10.3.36c)

and, for axis symmetrical signal spectrum

(10.3.37a)
430 Chapter 10

(1O.3.37b)

(J~ =0, (10.3.37c)

where (J! = No / AxL\y is variance of the input noise sampies,

(10.3.38)

is energy ofthe discrete signal {a k } ofthe target object and

(10.3.39)

are discrete inertia moments of the target object spectrum. One can obtain
them by the discretization offormulas (10.3.20), (10.3.22) and (10.3.23) and
using the relationship (10.2.9):

(10.3.40)

(10.3.41)
 10. Signal parameter estimation. Object localization 431

(10.3.42)

where N x = a/xöx is the number of signal sampies along coordinate

x , Ny =a/yay is the number of signal sampies along coordinate y,
{LV.. ,aly } are discretization intervals in the frequency domain and {a ,J is
2-D DFT spectrum ofthe target object signal a(x,y). It is assumed in Eqs.
10.3.40 - 42 that N .. and Ny are even numbers.

10.3.2 Illustrative examples

In order to gain an intuition conceming the relationship between the

signal shape and potential accuracy of its localization in additive Gaussian
noise, consider localization of a one-dimensional target object signals in
form of Gaussian shaped, rectangular and triangle impulses.
Let

(10.3.43)

be a Gaussian-shaped impulse. Its energy is:

Introduce an effective impulse width as the width of a rectangular impulse of

the same energy:

(10.3.45)

From Eq. (10.3.44) obtain that for Gaussian impulse:

(10.3.46)
432 Chapter 10

This relationship is illustrated in Fig. 10-6.

0.9

0.8 .•....•........ ,! ................ .

;
0.7 ......••.......• ; .....••••.•.•..

0.6 ................ 1. ............. .

0.5 ...........•.... 1................ .

0.4

0.3
0.2 ......•......... ; ............... .

0.1

.~ .!-5--::.0~.4--.-::"0.~3"'"""''-:.O~.,--.-::-O.1!-"--~--!:'-0.!-,--::0~.2......;;:;~0.3~--::0-!-.4--::-!0.5

Figure 10-6. Gaussian -shaped impulse and its efficient width shown by solid line. Dashed
line shows limits of interval [- er a,er a]

Spectrum of the Gaussian-shaped impulse is:

a (/) = -1a(x )exp(i 21tjX )tt = ao_1ex{ - 2: 2

: ) exp(i21tjX )tt =

•• lex{-( ~'; -i'bcfx + (fiaN} )}xp(v'ir,innf } dx =

., exp(- 2n '" ; f' )[exp[ - (;;'; - i'bcfx + (fi" .htf) )]dx =
(10.3.47)

and its inertia moment is:

(10.3.48)

Introduce also an effective spectrum bandwidth:

(10.3.49)
 10. Signal parameter estimation. Object localization 433

From Eqs.( 10.3.44), obtain:

2F = 1 (10.3.50)
a 2,Jna a

It follows now from Eqs.10.3.32,a and 10.4.44 through 47 that the

potential localization accuracy for the Gaussian-shaped impulse is
determined by equations

(l0.3.51,a)

or

(l0.3.51,b)

Für discrete signals, Eqs .10.3.51 may be written as:

(10.3.52, a)

and

a~_2 a~ _ 2 1
a
-2 -
da
I
V1[ a: - I
V1[ PSNR
2' (10.3.52, b)

where

PSNR=~. (10.3.53)
an
is the ratio of signal peak value to standard deviation of noise.
It is instructive to express, using Eqs. 10.3.46 and 10.3.50, the variance of
normal localization errors in terms of the effective impulse width and
efficient bandwidth of its spectrum:

(10.3.54)
434 Chapter 10

Thus, given energy of a Gaussian-shaped impulse, potential variance of

its localization normal errors is proportional to the squared effective width
of the target impulse and inversely proportional to the squared effective
bandwidth of its spectrum.
The relationship between impulse width and inertia moment of its
spectrum depends on the impulse shape. For rectangular impulses

a(x ) =aorect( S,:

x+s 12) (10.3.55)

one can numerically find that

f: == 0.2651 S" (10.3.56)

and, therefore for rectangular impulses

er; == 0,095S" No = 0.095 N: • (10.3.57)

E" ao

Similarly, for triangle impulses

a(x) = {2ao(3S" /2 + x)/3S",3S" /2 ~ x ~ 0

(10.3.58)
2a o(3S" /2 + x)/3S",0::;; x::;; 3S" /2

the following relationships hold:

2F" =E"/~(Ot =lIS" (10.3.59)

and

(-f:)l/1 == 0.183/ S", (10.3.60)

which is very elose to the corresponding relationship (Eq. 10.3.48) for

Gaussian-shaped impulses for which (f: )'1
s" == 0.1995.
Experimental verification of the performance of the described optimal
coordinate estimator is illustrated in Figs. 10-7 and 8. Fig. 10-7 shows
experimental distribution density of the localization error obtained by
computer simulation of localization of Gaussian-shaped impulse of width
 10. Signal parameter estimation. Object localization 435

Cf Q = 10 for 100000 realizations of noise with can JE IN

Q 0 = 1.5. One
easily distinguish in the plot the contribution to the error distribution density
of normal (central peak of the distribution density) and anomalous (uniform
tales ofthe distribution density) errors.
PrOb.bit!!,. d."sity 0' 11\. loeaUtellol'l ."0'; Sa.'O ; SN~.L5 , '00000

o .02fi

0 .02

0 _0' S

o 0'

0 .00.5

-'00 ·50 so , 00

Figure 10-7. Experimental distribution density ofthe localization error for Gaussian-shaped
impulse ( 0" a =10 ) and JE IN Q 0 = 1.5 obtained for 100000 realizations.

S10 of lne k)c: , 8'ror for GalJSsJan-shaped i ~ls8 o • = 10

4S

40 --. -. -.. --··, . - . ---. -.. .. .... --.. --- ..... -. -. --.. ...-. -..
~ ~ ~

· ... . .. ..
35 ··· ..
...
.0 __ • __ ._- ,_ •• _--_ ••••• _---_ •••• " •••• __ • • • • • • ___ ••••

30
· .
- - - - - - - - - - ·:- -- - - - - - - . - .. -- - - - - - -- -.- - -- _. .. . - . -... --.
··· .. ~

.. ....
~ ~.

.... --.. -... · - .... -. . . .... . .. .. . . . . . . ..... .. . ................

25
·· ... ... .. ..
.

20 - -- -------. · . . .
.
,----- --
·· .
- - - ,._----_._-
.. .----------".... ... _-_ ... -
.
15 •• -- •• -- •• . . • - --. To'tSI'std' 01 eirOiS' . :. ..... -- .. .
; -- -- • -- -- • : --
·· . .. ..
10 -------- - -.------_._.
·· ,..- _... . .. -... - ... _- -- --., .-.. . .. . .. .
· .. . . . .

oo.~"~'~'~'~"~'~'~'~;'~
0.1
'~'~":':'~'~"0~·~"='='~'~·~~1~~~~:n~'~:~~a~
.2 0.3
l e:rr~om~·tj':'~':"~'~'~'~'0']
0.4
'5
Sq<1{NOlEa)

Figure 10-8. Standard deviation of the localization error as a function of noise-to-signal error
for Gaussian-shaped impulse with (10000 realizations)

Fig. 10-8 illustrates how standard deviation of the localization error

depends on noise-to-signal ratio Nol E Graphs on the figure are plotted on Q •

the base of data obtained by the same computer simulation as that illustrated
436 Chapter 10

in Fig. 10-7. Lower curve shows standard deviation of errors smaller then
0'Q As one can see, this curve follows theoretical estimations described by
•

Eqs. 10.3.51 and 52. Upper curve shows standard deviation of the
localization error regardless their values. One can see that for low noise-to-
signal ratios localization error standard deviation behaves as it is predicted
by the theory for normal errors. However, for noise-to-signal ratio larger
then about 0.15, error standard deviation starts growing much more rapidly.
This behavior is associated with the appearance of anomalous errors.
Performance of the optimallocalization device in terms of anomalous errors
will be discussed in Sect. 10.3.4

10.3.3 Localization accuracy for non-optimallocalization devices.

Implementation of the correlator or matched filter assumes exact

knowledge of the shape of the target object. In practice, the shape is often
not precisely known or the matched filter cannot be precisely implemented.
In this section we estimate los ses in the localization accuracy owing to the
deviation of the real filter in the localization device from the matched filter.
We shall analyze these losses for the one-dimensional, or 2-D separable,
case.
Let the filter frequency response of the filter in the localization device
be H{Ix) instead of a ·(/J, the matched filter frequency response. The
output signal of such a filter will in this case be as follows:
<Xl

R(x) = Ja (Ix )H(tx )exp{- i21t lrx (x - Xo)]}tlx + Rnh (x), (10.3.61)

where Rnh ( x) is a result of filtering the white noise component in the

observed signal by the filter H(Ix). The location (xo + nx ) of the
maximum of this signal in the elose vicinity of the point X o (nx is small) is
defined by the equation:

~R(X~ =
8x 1x=Xo+nx

(- i:bt )-1/ .0: (JJH(J-Jexp{- i2x lr. (x - x. )]jd/.I~~_. +

~Rnh(X~ =0 (10.3.62)
8x 1x=Xo+nx
 10. Signal parameter estimation. Object localization 437

As nx is supposed to be small and, by definition, X o is the point of the

actuallocation ofthe object where the filter response is maximal,

00

(- i21t) flp. * (lx)H(lx)exp{- i21t [rx(x - xo)]~/x

(10.3.63)
00

41t 2nx f Ix2a *(Ix )H(lx }ilx

and

00

fixa *(/JH(lx}ilx = 0 (10.3.64)

Thus, we obtain :

(10.3.65)

Power spectrum of the random process :x Rnh(X)lx=xo+n x

is,

47t 2!x2NoIH(lxt. Then the variance (J ~ ofa small random error nx is:

00

N fl:IH(lxtdlx
2 0 _-=-",--00_ _ _ _ _-;:-
(10.3.66)
", ~ 4. ' (If:aVJHVJ1f, )' ,
or,

(10.3.67)

The third factor in this formula is the squared loss factor

 r
438 Chapter 10

LSFR~ (lf:w{j.l'df ·lf : W .l'df •

00
(10.3.68)
f/;a(/")H(/)I/,,
-00

that shows how much the normal error standard deviation for a non-optimal
filter exceeds that for the optimal (matched) filter: according to the Schwarz
inequality LSFR ~ 1. It reaches its minimal value, unity, when the filter is
perfectly matched to the object signal, that is when H(/,,) = a • (J,,).
In order to estimate the order of magnitude of the loss factor we present
results of analytical and numerical evaluation of the loss factor for
localization of Gaussian-shaped, rectangular and triangle impulses with the
help of the filters with, respectively, Gaussian, rectangular and triangle
impulse responses of different width.
For Gaussian-shaped impulses, matched

(10.3.69)

and mismatched

(10.3.70)

ones, one can analytically find that

2J3/2
LSFR = ( 1+0' mm /0' m
2
(10.3.71)
20' mm /0' m

This fimction is plotted in Fig. 10-9,a under the label "continuous

signals". A curve with the label "discrete signals" in this figure represents
results of a numerical evaluation of the loss factor for Gaussian-shaped
impulses of 1024 sampies with width parameters 0' m = 60 and
0' mm = 11 + 210. A certain difference between these curves can be
attributed to discretization effects. Fig.1 0-9, b) represents similar results for
localization of a rectangular impulse with SII =21 sampies of 256 with the
help of filters with a rectangular impulse response of different width. It is
instructive to see that while optimal localization of Gaussian-shaped impulse
 10. Signal parameter estimation. Object localization 439

is relatively tolerant to the inaccuracy of the matched filter width, it is very

sensitive to width mismatch between the rectangular impulses .

•..~ .......···t····· . ··!·· .. ···· .~ .......+- - ...~ _. - . ··1·········

: :J~~.::. :. :J:. :.."i::... ::..:-., :]': "
aQtI(LS R) ; : . ; 1 i

~ ......... ;......... 'f" ,...... .........

~ ,j, ....... ,
-f ..•••.•
: : : !i
3 S • ····+·········1- -.......;. ... ----.. : ..........~ ......... j ...• • •...

:~ ~ltJ~~~~
; . ;; i.l ·
10 ··,,···!,···,·,···;·,······1·_······~····· ··;··_·7(. ······ .. ~ ,·····,

o ~ ... ·+·· . ···,, !····· .. ··t. ··· . ·j··· .. ···"i .. ·"···+········ ' li' 'i;~l~
~.6 O~1
" 1 .~ 2 2 S
G." u laflo Im pu l at .... ldl,.. , . [10
0 .8 0.9 t
Sreel/SrO
1.1 1.2 1.3 t .4

a) b)

Figure 10-9. Loss factor for mismatched localization ofGaussian-shaped (a) and rectangular
impulses

10.3.4 Localization reliability. Probability of anomalous localization

errors

In this section we address the phenomenon of anomalous localization

errors. We defined anomalous errors as those that occur when the
localization device wrongly locates the object at output of the matched filter
outside the area occupied by the object signal. Because the noise at the
matched filter output results from filtering the white Gaussian noise it is
spatially homogeneous. Therefore, large noise outbursts may, with equal
probability, be found everywhere in an area outside the target object in is
actual location. This implies that the tales of the localization error
distribution density are uniform and one can characterize anomalous errors
by simply their total probability determined by the area under the
distribution density tales. This conclusion is supported by the experimental
distribution histogram ofthe localization error in Fig. 10-4.
Analytical evaluation of the probability of anomalous errors by
methods of the theory of random processes requires cumbersome
computation and can not be carried out without certain simplifying
assumptions ([7]). However its reasonably good estimation may be obtained
by the following simple reasoning.
Output of the matched filter outside the area occupied by the object is a
correlated Gaussian noise with correlation function defined by Eq. (10.3.4):

(10.3.72)
440 Chapter 10

Let tlS be a noise correlation interval, Le., a minimal distance at which

correlation between noise values can be regarded as negligibly small.
Because the correlation function of noise at the matched filter output is,
according to Eq. 10.3.72, proportional to that ofthe object signal, M has an
order of magnitude of the area occupied by the signal at the matched filter
output. Therefore, in the area of the search S there are approximately
Q = SI M uncorrelated sampies of Gaussian noise with variance
cr ~ = Rn (0,0)= NoR. (0,0)= NoE •.
Let us take one of this sampies in the point of actual location of the
object. According to Eq. 10.3.1, its value is equal to
R. (0,0)+ Rn = E. + Rn' where Rn is a Gaussian zero mean random value
with variancecr ~ . One can then find the probability of anoma1ous errors Pae
as a value complementary to the probability that none of the rest of (Q - 1)
uncorre1ated samp1es of Gaussian noise outside the object location exceeds
this va1ue:

where

(10.3.74)

is the "error integral".

Formula (10.3,73) known in communication theory as Kotelnikov's
integral ([2]) determines the probability of errors in communication channels
with Q orthogonal signal and additive white Gaussian noise ([5,6]). The
remarkab1e feature of this integral is its threshold behavior for 1arge Q .
Consider

(10.3.75)
 10. Signal parameter estimation. Object localization 441

for an auxiliary variable E = ~ E a / N 0 In Q . By L'Hospital's rule we have:

lirn
Q~co
[In[<t>( ~{E:
N"; +n)]Q-IJ= Q~co [~ln<t>(E~)/~!]=
lirn
'jdQ dQ Q

ifE 2 /2>t
(10.3.76)
ifE 2 /2st"

Therefore,

.
hrn P
Q~co ae
= {t, if E /2N o slnQ
a
0, if E a / 2N 0 > In Q
• (10.3.77)

This feature implies that if the area of search is large enough with
respect to the size of the object, the probability of anomalous errors may
become enormously high when input signal-to-noise ratio is lower than the
fundamental threshold defined by Eq. 10.3.77. This also means that when
area of search is increased the signal-to-noise ratio must also be increased in
order to keep the probability of anomalous errors low. Moreover, for any
given intensity of noise, there exists a trade off between localization
accuracy defined by the variance of normal errors and localization reliability
described by the probability of anomalous errors. Increasing the accuracy
achieved by widening the object signal spectrum with signal energy fixed
results in increasing the probability of anomalous errors: widening spectrum
width is equivalent to narrowing the signal, and, consequently, to increasing
the ratio Q ofthe area of search to the object signal area.
Results of a numerical verification of the above relationships by
computer simulation of the optimal localization device are shown in Fig.l 0-
10.
In conclusion it should be noted that the formula (10.3.77) has a general
fundamental meaning. Note that In Q may be regarded as the entropy, in
bits, of the results of measurements of object coordinate in which Q different
object positions can be distinguished. Therefore, Eq.(10.3.77) gives the
absolute lower bound for the object signal energy per bit of measurement
information required for reliable parameter estimation in the presence of
white Gaussian noise:

(10.3.78)
442 Chapter 10

Probability ofanomalous errors

0.8

0.6 I
Q=340
0.4

0.2
o
o 0.5 2 2.5

Figure 10-10. Probability of anomalous errors as a function of normalized noise-to-signal

ratio ~ No InQ! E Q for localization of rectangular impulses of 2;5; 11 ;21 ;41 ;81 ;161 sampies
within an interval of 1024 sampies (10000 realizations). Note that, according to Eq. (10.3.77),
the threshold value ofthe normalized noise-to-signal ratio is Ji/2 '" 0.707
 10. Signal parameter estimation. Object localization 443

10.4 LOCALIZATION OF AN OBJECT IN THE

PRESENCE OF ADDITIVE CORRELATED
GAUSSIAN NOISE

10.4.1 Localization of a target object in the presence of non-white

(correlated) additive Gaussian noise

In this section we extend the additive signal/noise model of Eq. (10.2.1)

IH" V"' /y rwhere IH" (Ix' f yr is a normalized spectrum shaping

to the case of non-white, or correlated, noise. Let the noise power spectrum
be No
function

(10.4.1)
-'"

such that noise variance is the same as that of the white noise with spectral
density No'
The problem of target localization in the presence of correlated noise
may be reduced to the white noise case if we pass the observed signal plus
noise mixture through a filter with frequency response

(10.4.2)

We will refer to this filter as to whitening filter. At the output of the

whitening filter, noise power spectrum becomes uniform with spectral
density No while the target object signal spectrum becomes equal to
a.Vx,/y)/ H"Vx,/J. Hence, in this case, ML-optimallocalization device
should consist of the whitening filter followed by a filter matched to the
target object signal at the whitening filter output and by the device for
localizing the signal maximum.
The whitening and the matched filters may be combined into one filter
with a frequency response

(10.4.3)

We will refer to this filter as to the optimalfilter.

444 Chapter 10

In this way we arrive at the optimal localization device shown in Fig.l O-

II that provides maximum likelihood (ML) estimation of the target object
coordinates .

................................................... ~

Object
Input
coordinates
image Whitening Correlator Device for
filter (matched localization of
filter) signal maximum

Whitening
Optimal filter
::rarget object signal
filter

Figure 10-11. Schematic diagram ofthe optimal device for localizing objects in the presence
of additive correlated Gaussian noise.

From the above reasoning it follows that, for a target signal observed in a
mixture with additive correlated Gaussian noise, the optimal filter generates
a signal which is proportional to aposteriori probability distribution TImction
of target coordinates. Now we will prove that the optimal filter has yet
another important feature. It also provides the highest possible for all linear
filters ratio of its response to the target object signal to standard deviation of
noise at its output (sig~al-to-noise ratio). This feature enables designing
optimal localization devices with linear filtering in case of non-Gaussian
signal observation model. We will use this feature in Ch. 11 for the design of
optimal devices for target location in clutter.
Consider response of an arbitrary filter with a frequency response
H{Ix'/y) to the target signal a(x-xo,y- Yo) and noise n(x) with spectral
density NoIHnVx,/yf. Filter output to the target signal at the point
(xo,Yo) is equal to

bo= _Lla Vx,/y)H(/x,/y)exP{i21t [/Jx- xo)+ /y(y- Yo)~d/xdfylx=Xo =

y=Yo
(10.4.4)
 10. Signal parameter estimation. Object localization 445

Standard deviation ofnoise at the filter output noise may be found as:

(10.4.5)

Their ratio, called signal-to-noise ratio, is then equal:

SNR = _ 0 =
b JJa {tx'/y"p{tx'/y}i/ß/y
-00-00 1/2 • (10.4.6)
er. [N.lllHV•• f,)'IH.V•• f,l'df.df, J
It is follows from Schwarz's inequality ([8]) that

JJa {tx' /y "p{tx' /y}i/xd/y =

11
-00-00

Y;:~!l [HV,. f,lH. V,. fJ~fßf, "

r r
;
1

{UI:.t::';,l;,df.df, OnHV,. f, 1'IH. V" f,)' df.df,

(10.4.7)

with equality taking place for the optimal filter of Eq.l 0.4.3. Therefore

(10.4.8)

10.4.2 Localization accuracy of the optimal filter

Because the optimal filter is reduced to the matched filter by the

whitening filter operation, the potential localization accuracy of the optimal
filter may be found from Eqs. 10.3.36 a-c for the localization accuracy in the
presence white noise appropriately modified to account for the whiting filter:

(10.4.9, a)
446 Chapter 10

-2
cr 2 = fx,NW NO. (10.4.9, b)
y,NW - 2 - 2 ( - 2 \2 4 2E '
fy,NW fy,NW - f xy,NW ) 1t a,NW

-2
2 fxy,NW NO (10.4.9, c)
cr xy,NW -,.2 /2 (j2 \2 41t 2E .
J y,NW y,NW - xy,NW) a,NW

where:

(10.4.10)

(10.4.11)

(10.4.12)

(10.4.13)

It is very instructive to compare the potential localization accuracy in the

cases ofwhite and non-white noise. Naturally, the comparison must be done
under certain normalization conditions. Let us compare the accuracy for the
same input noise variance. For the sake of simplicity we shall consider the
 10. Signal parameter estimation. Object localization 447

one-dimensional case. In this case we have from Eq.(IOA.9a) for white noise
that

(10.4.14)

Let us give this formula the form, which explicitly contains the error
variance (J ~ for the case ofwhite noise (Eqs. 10.3.37, a):

-2
2 1 No = er 2 Ealx
er = (10.4.15)
x,NW -2 2 x E 1'2
Ix,NW 411: Ea,NW a,NWJx,NW

The last factor in this formula shows the change of error variance owing
to non-whiteness of noise. Because all functions involved in this factor are
nonnegative, the following inequality takes place:

(10.4.16)

Therefore, the potential localization accuracy in the presence of non-

white noise is always better then that for white noise with the same variance.
Note that in the trivial special case when noise is band limited and its
bandwidth is less that that of the signal, optimal filter is a band pass filter
which lets through all input signal frequencies where the noise spectrum
vanishes to zero. The resulting signal is therefore noise free and the potential
localization error variance is equal to zero.
In order to make a numerical evaluation of the decrease of the
localization error variance for non-white noise, consider a one-dimensional
Gaussian-shaped signal
448 Chapter 10

(10.4.17)

observed in a mixture with Gaussian noise with power spectrum:

(10.4.18)

In this case optimal filter frequency response is

(10.4.19)

where Sa = J;cr a is the impulse efficient width (Eq. 10.3.46). One can
see from Eq. 10.4.19 that optimal filter is in this case a sort of a band-pass
filter with central frequency 1/ 4nS; Fn • A family of its frequency responses
is illustrated in Fig. 10-12.
Opl lmaf fllI., frequent)' ' •• ponu

0 .9
0 .8

0.7
0 .6

Frequency "lEffic lenl lmpul •• w idtll

Figure 10-12. Frequency responses of the optimal filter for Gaussian-shaped impulse
2S;)
a(x) = exp(-n x 2 / and non-white noise with exponentially decaying power spectrum

INnVt =exp(-Ifl/ FJ,k = SaFn;

10. Signal parameter estimation. Object localization 449

10.5 OPTIMAL LOCALIZATION IN COLOR AND

MULTI COMPONENT IMAGES

10.5.1 Optimallocalization device

The results presented above for optimal localization in monochrome

images may be extended to localization in color, or, generally, multi-
component images, observed in the presence of additive Gaussian noise in
each component.
Let {ak,,,,(xo'Yo)}' k=O,l, ...,N-l, be m-th component of the target
object signal, M be the number of components m = 1,2,•.. ,M (for color
images M=3). Let also {Hk,,,,} and {bk,,,, } be the sampies of the
corresponding components of the additive Gaussian noise and of the
observed signal, such that

(10.5.1)

Let also assume that components of the additive noise as weIl as sampies of
noise within each component are aIl mutually uncorrelated. This assumption
directly implies, that probability of observing signal {bk •m } provided target
object is located in coordinates (xo,Yo) is

where cr ~.m denotes the variance of the m-th noise component. Therefore,
optimal MAP- and ML-Iocalization devices are defined, respectively, by
equations

and
 450 Chapter 10

(10.5.4)

For continuous signals, MAP-estimation is

where N O,m is the spectral density of the m-th noise component, and ML-
estimation is

Therefore optimal localization device should consist of M parallel

component wise correlators, or matched filters, of an adder for a weighted
summation correlator outputs and of a unit for determining coordinates of
the signal maximum at the adder's output. Its schematic diagram is shown in
Fig. 10-13.

Matched filter
(I-st component)

Input
Matched filter
image (2-nd component)
Device for
components
Adder localization of
Matched filter signal
(3-d component)
maximum

Matched filter Object's coordinates

(M-th component)

Figure J0- J3. Schematic diagram of the optimal device for localization of a target object in
multi component images
10. Signal parameter estimation. Object localization 451

10.5.2 Localization accuracy and reliability

Variances of norma11ocalization errors, that characterize the accuracy of

optimal localization, may be found with just the same technique as it was
done for single-component images in Sect. 10.3.1. In this way one can obtain
([7]):

(10.5.7)

(10.5.8)

where

(10.5.10)

(10.5.11)

M 1
L ~ I I/y1rV",/yf d/"d/y
0000

f-y,ll1 -- ",=1
M
0,'"
1
-00-00

00 00 1
(10.5.12)
~~ I IrV",/y~ d/"d/y
1ft- 1 0,,,, -00-00

M 1 0000 1

L~ I I/"/yrV,,,/y~ d/"d/y
J:-xy,a
1 - ",=1 0,'" -00-00
(10.5.13)

L ~ I Ir v"' /y ~ df"d/y
- M 1 00 00 1

111=1 0"" -00-00

452 Chapter 10

(1.6.14)

(10.5.15)

(10.5.16)

where

(10.5.17)

cr;,M = LNo,m f ff(Jx,fyrdfxdfy.

M "''''0
(10.5.18)
m=1

In the case ofthe signal symmetry, when l!,M = 0,

0' 2 = Yx 0' n,M •
2
0' 2 = _Yy _ 0' 2
n,M • 0' 2 = 0. (10.5.19)
x,M f2 4n 2 E 2 ' y,M 1'2 4n 2 E2 ' xy,M
x,M II,M J y,M a,M

As for the probability of anomalous errors, it can be found from Eq.

(10.3.73) with substitution ofthe involved values Eil and 0' n by EII,M and
0' n,M , respectively.

10.5.3 Optimallocalization in multi component images with correlated

noise.

As in the case of single component images, for localization in multi

component images, when noise components are correlated, input image and
target object image pre-whitening may be used in order to reduce the
problem to the previous one. The optimal localization device of Fig. 10-3
should then be modified as it is shown in Fig. 10-14.
 10. Signal parameter estimation. Object localization 453

-. -+ Matched filter ....

(I-st component)

-. f--+ Matched filter

......

..
(l-nd component) Device for
localization
Pre- -+ Adder
ofsignal
"whitening" maximum

..... filter
~
Matched filter
(3-d component)
...
po

f
.... +
Object's
coordinates

..... r--+
Matched filter
(M-th component)
......

+-
r----- -

I , f
Pre-whitening filter

f ~
Target image components
,I
Figure 10-14. Schematic diagram ofthe optimal device for localizing objects in multi-
component images with additive correlated noise in the channels.

Frequency response of the pre-whitening filter is defined in the 3-D

frequency domain as

(10.5.20)

where {Ix' I y } are spatial frequencies, cr is an index of component-wise

Fourier Transform (for multi-component images, the latter is the component
wise Discrete Fourier Transform) and N(a ,Ix' /J
is a spectral density of
noise defined in the 3-D frequency domain.
454 Chapter 10

10.6 OBJECT LOCALIZATION IN THE PRESENCE OF

MULTIPLE NONOVERLAPPNING NON-TARGET
OBJECTS

In this section, we discuss an image model in which observed image

signal b(x,y) contains, along with the target object signal a(x- xo'y- Yo)
in coordinates (xo,Yo) and additive white Gaussian sensor noise n(x) ,
some number Qe ofnon-target objects {aq(x-xq,y- Yq)} with coordinates
{(xq,Yq)}:

b(x,y) =a(x - xo'Y - Yo)+ taq (x - xq'Y - Yq)+ n(x,y). (10.6.1)

9=1

Suppose also that the non-target objects do not overlap one another and the
target object. The most immediate illustration of such a situation would be
the task of locating a specific character in a a printed text.
As non-target objects do not overlap the target object under, it is clear
that the design ofthe filter ensuring the highest localization accuracy, i.e. the
lowest variance of normal errors, is govemed by the same reasonings as
those for above described additive noise model with no non-target objects.
Therefore optimal filter in this case will also be the matched filter and the
variance of normal errors will be defined by the same formulas as those in
the cases of no non-target objects (Eqs. 10.3.34 and 10.3.36). Presence of
non-target objects has an effect only on the probability of anomalous errors.
In order to estimate the probability of anomalous errors in target
localization in the presence of multiple non-target objects let us introduce
some quantitative characteristics of the non-target objects. Let
• p(Qe) be the probability of appearance of exactly Qe non-target objects
in the area of search.
• Non-target objects form some C c1asses, such as, for instance, c1asses
of characters in optical character recognition, according to the maximal
values R!~ of their cross correlation function with the target object

(10.6.2)

c = 1,2,•••,C
 10. Signal parameter estimation. Object localization 455

• P(Q.,I ,Q.,l ,••••Q.,c) be the joint probability of appearance of Q.,I

objects of the first class, Q.,l objects of the second class and so on,
given the total number of false objects is Q•.
Obviously, in the presence of non-target objects anomalous errors will
occur mainly as a result of false localization of the target object at the
location of one of the non-target objects. Signal values at these are equal to
maxima of the matched filter responses to the non-target objects, i.e., to
cross-correlations (Eq. 10.6.2) between the target object and the non-target
objects, plus correlated noise resulting from the additive noise at the
matched filter input. Then the probability of anomalous errors may be
evaluated as being the complement to the probability that all signal values at
the output of the matched filter at locations of non-target objects do not
exceed the signal value at the location of the actual location of the target
object. As non-target objects do not overlap one another, the values of
additive noise at the points of location of non-target objects may be regarded
uncorrelated. Therefore, one can, by analogy with Eq.(I0.3.73), evaluate
probability of anomalous localization errors (probability of false detection)
as

(10.6.3)

where as before

(10.6.4)

It is convenient to represent Eq.lO.6.3 through the appearance

probabilities {Pe (c)} of non-target objects of the class c making use the fact
that the probability distribution of the appearances of exactly
{Q.,I ,Q.,l ,··••Q.,c} of each class non-target objects from Q. is polynomial
and is connected with the probabilities {p. (c)} by the following equation:

(10.6.5)
456 Chapter 10

After substitution of Eq. 10.6.5 into Eq. 10.6.4 and with the use of the
identity

(10.6.6)

we obtain:

(10.6.7)

Thus, for estimating the probability of anomalous errors, it is necessary

to know noise spectral density No, (or, for signals in discrete representation,
noise variance) at the input of the matched filter, target object signal energy
E a , differences {E
a - Rt~} between auto and cross-correlation peaks,

probabilities {Pe (c)} of non-target objects of different classes, and

probability p(Q,) of observing in the area of search exactly Q, non-target
objects.
One can obtain more tractable formulas and practical recommendations
by considering the following special cases.
1. Let probabilities p(Q,) are concentrated within a relatively narrow

range [Q"min,Qe,max] ofvalues of Q,: (Qe,max -Qe,min)IQe,min« 1. For

instance, in case of optical character recognition, this assumption is
equivalent to the assumption that variations of characters that belong to the
same class are smalI. Then for Q"min ~ ß ~ Q"max we obtain an estimate:

(1- Qe,max - Qe,mln

Qe,min
Jpa
(Q )< P <
\1 e,O - a -
[1 + Qe,max - Qe,mln
Q
e,min
Jp (Q
a \1
) (1068)
e,O' ••

where

(10.6.9)
 10. Signal parameter estimation. Object localization 457

Therefore a good estimate of the probability Pa of anomalous errors in

this case is the value Pa (Qe,o) at some intermediate point Qe,O.
2. The probability distribution p(Qe ) is concentrated around some
point Qe,o and there is only one class of the non-target objects (C = 1). In
this case

This expression is similar to Eq. 10.3.73 for the probability of anomalous

errors in the absence of non-target objects. The only difference is that now
signal-to-noise ratio is substantially reduced by the cross-correlation peak of
the non-target and target objects.
Obviously, the probability of anomalous error also features threshold
behavior:

(10.6.11)

For instance, for Q = 16 to 2048 and

E - R(c) ( . )
~ == 1.1-1.6 JlnQo == 1.8-4.4. (10.6.12)
E
a 0
N· .

Pa,e < 10-2 and it increases rapidly with decreasing signal-to-noise ratio
E
Wo .
_R(c)

EaN o
Therefore the probability of anomalous errors in the presence of

multiple false objects may be very high. Note, for example, that the number
of characters on a standard printed page is about 1600. Fig. 10-15 illustrates
the phenomenon of false detection on an example of character recognition.
In order to decrease Pa,e' it is necessary to increase signal- to-noise ratio
by suppressing cross-correlation peaks for non-target objects with respect to
the auto-correlation peak of the target object. This requires an appropriate
modification of the matched filter. Therefore, for localization of a target in
 458 Chapter 10

Figure 10-15. Detection, by means of matched filtering, of a character with and without non-
target characters in the area of search. Upper row, left image: noisy image of a printed text
with standard deviation of additive noise 15 (within signal range 0-255). Upper row, right
image: results of detection of character "0" (right); one can see quite a number of false
detections. Bottom row, left: a noisy image with a single character "0" in its center; standard
deviation of the additive noise is 175. Highlighted center of the right image shows that the
character is correctly localized by the matched filter.

the presence of non-target objects, it is not possible to simultaneously secure

the minimum of normal error variance and the minimum of the probability
of anomalous errors with the same estimator and optimal localization must
be carried out into two steps. The first step is target detection with minimal
probability of anomalous errors. A coarse but reliable estimate of the object
coordinates to the accuracy ofthe object size is obtained at this stage. For the
accurate estimation of target coordinates, the second step is needed in which
localization with minimal variance of normal errors is obtained in the area of
search in the vicinity of the location found at the first step. The optimal
localization device for the second step is the matched filter correlator. The
problem of the design of optimal device for reliable detection of targets on
the background of a clutter of non-target objects is addressed in eh. 11 .
10. Signal parameter estimation. Object localization 459

References
1. A.Van der Lugt, Signal Detection by Complex Spatial Filtering, IEEE Trans., IT-IO,
1964, No. 2, p. 139
2. L.P. Yaroslavsky, M.G. I1ieva, V.N. Karnaukhov, I.D. Nikolov, G.1. Sokolinov, A
Device for Pattern Recognition, Certificate of Authorship No 1414170, of 24.4.1986,
Reg. 01.04.1988
3. C. S. Weaver and J. W. Goodman, " A technique for optically convoluting two
functions," Applied Optics, 5, pp. 1248-1249 (1966).
4. F. T. S. Yu and X. 1. Lu, "A real-time programmable Joint Transform Correlator", Opt.
Commun., Vol. 52,1984, p. 47.
5. V. Kotelnikov, The Theory of Potential Noise Immunity, Moscow, Energia, 1956 (In
Russian)
6. J. M. Wozencraft, I. M. Jacobs, Principles ofCommunication Engineering, N.Y.,Wiley,
1965
7. L.P. Yaroslavsky, The Theory of Optimal Methods for Localization of Objects in
Pictures, In: Progress in Optics, Ed. E. Wolf, v.XXXII, Elsevier Science Publishers,
Amsterdam, 1993
8. L. Schwartz, "Analyse Mathematique", Hermann, Paris, 1967
 Chapter 11

TARGET LOCATING IN CLUTTER

11.1 PROBLEM FORMULATION

In this chapter we discuss the problem of locating targets in images that

contain, besides the target object, a clutter of non-target objects that obscure
the target object. As it follows from the discussion in Sect. 10.7, background
non-target objects represent the main obstacle for reliable object localization
in this case. Our purpose therefore is to find out how can one design a
localization device that minimizes the danger of false identification of the
target object with one of the non-target objects.
A general solution of this problem similar to that described in Ch. 10 for
the model of a single target object observed in additive signal independent
Gaussian noise does not seem to be feasible in this case for two reasons.
First, the least bit constructive statisticalor whatever description of image
ensemble is very problematic owing to the great variability of real life
images. Second, and this is even more important, in applications where
target location in images with cluttered background is required one is
interested in the most reliable localization not on average over a hypothetical
image ensemble but on each individual image. Therefore, optimization and
adaptation of the localization algorithm for individual images are desired.
This requirement of adaptivity will be imperative in our discussion.
The second imperative requirement will be that of low computational
complexity of localization algorithms. Bearing this in mind we shall restrict
the discussion with the same type of localization devices as those described
in Ch. 10, i.e., the devices that consist of a linear filter followed by a unit for
locating signal maximum at the filter output (Fig. 11-1). Owing to the
existence of fast and recursive algorithms of digital linear filtering (Chs. 5,
6), such devices have low computational complexity in their computer
462 Chapter 11

implementation. They also have promlSlng optical and electro-optical

implementations. In such type of the devices, it is the linear filter that has to
be optimized and is the subject of the adaptation.

Input Object's
image Device for coordinates
... Linear filter ... locating .....
b(x,y) b'(x,y) signal maximum

Figure 11-1. Schematic diagram ofthe localization device

We will begin the discussion with a fonnulation of the notion of

optimality using an illustrative diagram of Fig. 11-2. Let us assume that
input image b(x,y) with a target object in coordinates (xo,Yo) may be
decomposed into S fragments of areas {SJ, s = 1,•••,S that may be
regarded homogeneous in tenns of the price for false localization errors
within these areas. Let also hs(bbg I(xo,yo)) be a histogram ofsignal values
b(x, y) at the filter output as measured for the s-th fragment over
background points not contained within the target object and bo is the filter
output at the object location.

Figure 11-2. Illustrative diagram ofbasic notations

 11. TARGET LOCATIng IN CLUTTER 463

The localization device under consideration makes its decision regarding

the coordinates of the desired object upon the position of the highest
maximum at the signal output. Consequently, the integral

(11.1.1)

where AVbg symbolizes averaging over an ensemble of the background

component of the image, represents the rate of the s-th fragment points that,
on average (over sensor's noise and different possible background objects),
can be erroneously assigned by the decision unit to the target object.
Generally speaking, linear filter output at the target object Iocation bo
should also be regarded a random variable because it may depend on signal
sensor noise, photographic environment, illumination, object orientation,
neighboring objects, and numerous other factors unknown for the filter
design. In order to take this uncertainty into the consideration, introduce a
function p(bo ) that describes apriori probability density of bo • Unknown
object coordinates (xo,Yo) should also be regarded random. Moreover, the
price of the measurement errors may differ over different images fragments.
To allow for these factors, we introduce weighting functions w.(xo,Yo) and
JJ: characterizing the apriori significance of the localization within the s-th
fragment and for each s-th fragment, respectively, as :

ffw.(xo,yo}ixodyo = 1; (11.1.2)
s.

s
LW, =1.
s=l
(11.1.3)

Then the quality of estimating coordinates by the device of Fig. 11-1 may
be described by a weighted mean with respect to p(bo)' {w.(xo,Yo)}' and
{W,} ofthe integral ofEq.(11.1.I):

1'. ~lp(b,)db,tw, IJ""(x"y,{ Av.. lh,(b.. /(x"y,)~b r'dy,

(11.1.4)
464 Chapter 11

A device that provides the minimum for the Pa will be regarded as

optimal on average with respect to sensor's noise and background
components of the image. For the design of the adaptive device optimal for a
fixed image, one should drop out from the above formula averaging
AVbg over the background.
 11. TARGET LOCAT1ng IN CLUTTER 465

11.2 LOCALIZATION OF PRECISELY KNOWN

OBJECTS: SPATIALL Y HOMOGENEOUS
OPTIMALITY CRITERION

11.2.1 Optimal adaptive correlator

Assume that the target object is defined precisely. In this context, this
means that the response of any filter to this object may be determined
exactly. Therefore, probability density p(b o ) is the delta-function:

(11.2.1)

where bo is the known filter response at the point of the target object
location. Then Eq. (11.1.4) that defines the localization quality becomes:

Denote the histogram averaged within each fragment over (x o, Yo) by

(11.2.3)

Then obtain:

(11.2.4)

Suppose that weights {Ws} do not depend on sand are equal to IIS. We
refer to this case as to the case of the spatially homogeneous optimality
criterion. Then

-(-) 1 ~ - (- ) (11.2.5)
h \bbg = - L.. h s \bbg
S s=l
is the histogram of the filter output signal amplitudes as measured over the
background part of the whole image and averaged with respect to the
466 Chapter 11

unknown coordinates of the target object. By substituting Eq.(l1.2.5) into

Eq.(11.2.4), we obtain

P" = jAVbg{Ii{b)~b. (11.2.6)

bo

Let us now determine the frequency response H(Jx' I y ) of a filter that

minimizes p". The choice of H(Jx' I y ) affects both bo and histogram
Ii(bbg). As bo is the filter response at the target object position, it may be
determined through the target object Fourier spectrum Cl (Ix' I y ) as

Ja {1x,J;, )H{Ix,t, )dlxdt,.

CI)

ho = (11.2.7)
-CI)

As for the relationship between Ii(bbg )and H(Jx' I y )' it is, generally
speaking, of a more involved nature. The explicit dependence of Ii(bbg) on
H(/x '/y) may only be written for the second moment m; of the histogram
li(b) using Parseval's relation for the Fourier transform:

m; = Hw(xo,yo)dxodyo 1bb~h(bbg l(xo,Yo)}tbbg =

S

Sl Hw(xo,Yo)dxodyo Jbb~(X,y}dxdy=
bg S Sbg

-f-bg Hw(xo,yo)dxodyo
S
11Iß1;·O)(lx,ly fI H(Jx,ly f dlxdly =

111 ß!;.O) (Ix' I rIH(Jx' I f d/xd/

-00-00

y y y , (11.2.8)

where S bg is the area of the background part of the images, or the image
search area minus the area occupied by the target object located at
coordinates (xo,Yo)' ßb~·O)(JX,/J is Fourier spectrum ofthis background
part, and
 11. TARGET LOCAT1ng 1N CLUTTER 467

(11.2.9)

Therefore, for the optimization of the localization device filter we will rely
upon the Tchebyshev's inequality

Probability(x ~ x + b o) ~ (J 1 I b; . (11.2.10)

that connects the probability that a random variable x exceeds some

threshold bo and the variable's mean value x and standard deviation cr .
Applying this relationship to Eq.(11.2.6), we obtain:

(11.2.11 )

where b is mean value of the histogram h(b ). By virtue of the properties of

the Fourier Transform the latter can be found as:

(11.2.12)

It follows from this equation that the mean value of the histogram over the
background part of the picture, b, is determined by the filter frequency
response H(0,0) at the point (Jx = 0, /y = 0). This value defines a constant
bias of the signal at the filter output which is irrelevant for the device that
localizes the signal maximum. Therefore, one can choose H(O,O) = 0 and
disregard b in (11.2.11). Then we can conclude that, in order to minimize
the rate p" of anomalous errors, a filter should be found that maximizes
ratio of its response bo to the target object to standard deviation (m: )1/2 of
its response to the background image component

b JJa (Ix' /y )H(Ix' /y }i/xd/y

..: LU
SCR = ( =

(r~· v.. !, l' )HV..!, l' dU!, J

)1/2 -"J-"J 1/2

A V ..

(11.2.13)
468 Chapter 11

We will refer to this ratio as to s;gnal-to-clutter ratio.

It follows from Schwarz's inequality ([1]) :

(11.2.13)

that frequency response of such an optimal filter is defined by the equation:

(11.2.14)

where asterisk * denotes complex conjugation.

One can see that filter defined by Eq. 11.2.14 is analogous to the optimal
filter (Eq. 10.4.3) introduced in Sect.10.4 for object localization in
correlated Gaussian noise. The numerator of its frequency response is the
frequency response of the filter matched to the target object. The
denominator of its frequency response is power spectrum
AVbg (/a :;0 (Jx ,I, ~ 2) of the image background component averaged over

an ensemble of images that may contain the same target. It plays here the
role additive Gaussian noise power spectrum plays in the filter ofEq. 10.4.3.

/a :;0 (Jx' I, r
There is, however, an important difference. When averaging is applied to
the filter will be optimal (in terms of minimization of the
misdetection rate) also on average over the ensemble of possible images.
However, as we mentioned in Sect. 11.1, in applications, one frequently
needs a filter optimal for each particular image rather then on the average.
This means that in this case the averaging over the background image
component AVbg must be dropped out from the optimality equation 11.2.13.
In this way we arrive at the following filter:
 11. TARGET LOCAT1ng IN CLUTTER 469

(11.2.15)

This filter will be optimal for the particular observed image. Because its
frequency response depends on the image background component the filter
is adaptive. We will call this filter "optimal adaptive correlator".
To be implemented, optimal adaptive correlator needs knowledge of
power spectrum of the background component of the image. However
coordinates of the target object are not known. Therefore one cannot
separate the target object and the background in the observed image and can
not exact1y detennine the background component power spectrum and
implement the exact optimal adaptive correlator. Yet, one can attempt to
approximate it by means of an appropriate estimation of the background
component power spectrum from the observed image.

11.2.2 Implementation issues, practical recommendations and

illustrative examples

There might be different approaches to substantiating spectrum

estimation methods using additive and implant models for the representation
of target and background objects within images. ([2,3]). In the additive
model, input image b(x, y) is regarded as an additive mixture of the target
object a(x - xo' y - Yo) and image background component abg (x, y):

(11.2.16)

where (x o,Yo) are unknown coordinates of the target object. Therefore

power spectrum of the image background component averaged over the set
of possible target 10cations may be estimated as

r!:,O){Ix,fyf =Iß{Ix,fyf +r{Ix,fyf +

ß •{Ix' fy)x (fx' f y~xp[i21t Vxx o+ fyYo )]+
ß{Ix' fy)x •{Ix' f y~xp[- i21t Vxx o+ fyYo ], (11.2.17)

where the horizontal bar denotes averaging over target coordinates (x o,Yo).
470 Chapter 11

Functions exp[- i21t (fxxo + fyYo )] and exp[i21t (fxxo + fyYo )] are Fourier
transforms ofthe distribution density p(xo,Yo) ofthe target coordinates:

Xy

( 11.2.18)

In the assumption that the target object coordinates are uniformly distributed
within the input image area, nmctions exp[- i21t (Ixx o + lyYo )] and
exp[i21t (Ix X o + I yYo )] are sharp peaks with maximum at Ix = I y =0 and
negligible values for all other frequencies. Therefore, for the additive model
of the target object and image background component, averaged power
spectrum ofthe background component may be estimated as:

(11.2.19)

The implant model assurnes that

(11.2.20)

where w(x - x o' Y - Yo) is a target object window function:

( )= {
wx,y
o within the target object . (11.2.21)
1 elsewhere

In a similar way as it was done for the additive model and in the same
assumption of uniform distribution of the coordinates over the image area,
one can show that in this case power spectrum of the background image
component averaged over all possible target coordinates may be estimated as

where W (Ix' I y) is Fourier trans form of the window function w(x, y). This
method of estimating background component power spectrum resembles the
conventional methods of signal spectral estimation that assurne convolving
 11. TARGET LOCATIng IN CLUTTER 471

power spectrum of the signal with a certain smoothing spectral window

function W(Px,pJ ([4]):

(11.2.23)

or, in the discrete representation,

r:;O{r,st == LL W{rw,swJß{r - rw's - Sw t (11.2.24)

'W 'W

Both models imply that, as a zero order approximation, input image power
spectrum may be used as an estimate of the background component power
spectrum:

(11.2.25)

This approximation is justified by the natural assumption that the target

object size is substantially smaller then the size of the input image and that
its contribution to the entire image power spectrum is small with respect to
that of the background image component.
Experimental comparison of these approaches showed that the spectrum
smoothing method provides better and more stable and image independent
results in terms of the achievable signal-to-clutter ratio. Experimental data
show also that a rectangular window may be used for image spectrum
smoothing and that the window size is not very critical. Practical
recommendations are rectangular window functions of 5x5 to l5x 15 pixels.
In the conclusion of this section we illustrate advantages of the
optimal adaptive correlator over the conventional matched filter
correlator that is optimal for target location in Gaussian noise in the
absence of non-target objects and clutter.
Fig.11-3 compares matched filter correlator and optimal adaptive
correlator for localization of the corresponding image fragments in
stereoscopie images. This comparison shows that the matched filter is
incapable of reliable locating small image fragments of one of the
stereo images in the second image while the optimal adaptive
correlator does substantially suppress the background image
component and therefore has much better discrimination capability.
472 Chapter 11

The result of locali2lltion (marked with a cross) Target (highlighted)

Output ofthe correlalor (row) Output ofthe correlalor (column)

,
1.6
( "\ ! Y'
1.4
:\ /I /1'.
1.6
I ! J fi ~
f iI I l \ i-/+-- \
~\
I. '\, ,..,11 ,
1.4 I !
1.2 , oJ I I !I, i /\ i
I u
1.2
i I I \I
0.8

,,
I
' I I J ~ ['[--J I I
0.6
\)~- 0.8
,/, I i
50 100 150 200 250 20 40 60 80 100 120

a)

The result of localization (marlced with a cross) Ta'XeI (highlighted)

OUlpUt ofthe orrelator (row) Output ofthe cOITelator (column)

~o 100 1SO 200 250 20 40 60 80 100 120

b)

Figure 11-3. Localization of correspondent points in stereoscopie images with the matched
filter (a) and the optimal adaptive correlators (b)
 11. TARGET LOCAT1ng IN CLUTTER 473

In Fig. 11-4 data are presented of an experiment in whieh a round spot

of about 32 pixels in diameter was randomly plaeed within the image
of ehest X-ray of 512x512 pixels. The initial X-ray and outputs of the
matehed filter and optimal adaptive correlators are shown in Fig. 11-4,
a) in form of gray scale images and graphs of signal rows that go
through the spot center in one of its positions. The shape of the spot is
illustrated in 3-D plot of Fig. 11-4, b) in coordinates gray scale value
versus spatial coordinates.
Average rates of false localization found for different contrast of the
spot over the ensemble of the spot random positions uniformly
distributed within the image are plotted in graphs of Fig. 11-4, c) in
coordinates rate of misdetection versus ratio of the spot maximum to
standard deviation of the image gray scale values.. These results
clearly show that, with the optimal adaptive filter correlator, the spot
can be reliably detected for much lower contrast that with the matched
filter correlator.

O utput ofthC matchcd filtcr

l
Init ial signal Output ofthc optimal COrrclator
220 ...r\ j\.

,I,;'..1
1 !
\; 02

,.
6.5
200ifl \ "
t\ 0.1 I

::,'
6 ii
:l \

j , j \ ,,tl J-y/\!V-\i\t ,.f.,,~

i
'I
I 5.5 ' i
,
~.; I

\ ! j
\........ /
i ~.~I V 'i'"
I
.. ! 5
( 1

140 \wS 'll/' 4.5 J

tOO 200 300 400 500 100 200 300 400 500 100 200 300 400 500

0.8

0.6

0.4

0.5 I 1.5 2 2.5 3

b) c

Figure 11-4. Comparison of localization reliability of the matched filter and optimal adaptive
correlators: a) a test X-ray image and outputs ofthe matched filter and ofthe optimal adaptive
correlators; b) - 3-D plot of the target test spot; c) - plots of false detection rates versus test
spot contrast ratio to the image standard deviation for the matched filter correlator (Jeft) and
fOT the optimal adaptive correlator (right).
474 Chapter 11

11.3 LOCALIZATION OF INEXACTLY KNOWN

OBJECT: SPATIALLYHOMOGENEOUS
CRITERION
Optimal adaptive correlator introduced in Sect. 11.2.1 is tuned to a target
object that is exactly known. However in reality there might be some
uncertainty about target object parameters such as orientation, scale, other
geometrical distortions, distortions caused by sensor's noise, etc. In these
cases the probability density p(bo ) of the filter response to the target object
in its coordinates can no longer be regarded to be a delta function. Therefore
the filter ofthe optimal estimator must secure the minimum ofthe integral:

(11.3.1)

where ii(bbg) is defined by Eq.(11.2.3).

Two options to reach this goal may be considered, depending on
implementation restrictions.

8. Localization device with multiple filters. Split the interval of possible

values of bo into subintervals within which p(bo)may be regarded to be
constant. Then

(11.3.2)

where b}') is a representative of q-th interval, P, is the area under

-
p(b o ) over the q-th interval and Q is the number of the intervals.
Because p k ~ 0, p.. is minimal, if
(11.3.3)

is minimal and the problem reduces to the above problem of localizing a

precisely known object. The only difference is that now one should consider
Q different versions of exact1y known target object. Therefore, optimal
filters
 11. TARGET LOCATlng IN CLUTTER 475

(11.3.4)

need to be generated separately for each of Q "representative" of all possible

object variations of the target object. This solution is, of coarse, more
expensive in tenns of the computational or hardware implementation
complexity than that with a single filter.
b . Localization device with optimal-on-average filter. If variances of
the target object variable parameters are not too large, one may, at the
expense of worsening the localization reliability, solve the problem with a
single filter as though the target object were exactly known. The optimal
filter in this case has to be corrected with due regard to the object parameter
variations. In order to find the frequency response of this filter, change the
variables ~g ~ b = ~g - ho and change the order of integration in Eq. 11.3.1:

(11.3.5)

The internal integral in Eq. 11.3.5, is a convolution of distributions, or

distribution of the difference of two variables band ho. Denote this
distribution by hp (b -hu ) . Its mean value is equal to the difference of mean
values of ho p() and h (), respectively and its
and bav of distributions
variance is equal to the sum [m; - (bav YJ+ (J; of variances of these
distributions, where cr; is the variance of the distribution p(.). Therefore,

(11.3.6)

Thus, the problem has again been boiled down to that of Sect.11.2, and
one can use Eq. 11.2.14, with corresponding modifications of its
numerator and denominator, to detennine the optimal filter frequency
response. As 'ho is the mean value of the filter output at the target object
location over the distribution p(ho) , a complex conjugate spectrum
u*Vx,tJ of the object in the numerator ofthe fonnula (11.2.14)
should be replaced by a complex conjugate of object spectrum u* Vx' t y)
476 Chapter 11

averaged over variations of the target object. As for the denominator of Eg.
11.2.14, it should be modified by adding to it the variance of the object
spectrum

(11.3.7)

where AVobj (.) denotes the operator of averaging over variations of the
target object uncertain parameters. As a result, we obtain the following
equation for the "optimal-on-average filter" frequency response:

(11.3.8)

This filter will be optimal-on-average over all variations of the target

object and over all possible backgrounds. By eliminating averaging over the
background we obtain the optimal-on-average adaptive filter:

(11.3.9)

which will be optimal-on-average over variations of the object for a fixed

background; that is for the given observed image. For the design of this

r:;0 (Ix'
filter, methods of estimating background image component power spectrum
/y )Zdiscussed in the previous section may be used. In a special
case when variations of the target object are caused by additive sensor' s
noise of the imaging system, the variance of the object spectrum is equal to
the spectral density of the noise, and optimal filter becomes:

(11.3.10)

where ~ (Ix' /y r denotes the noise spectral density. As it was already

mentioned, such an averaged filter cannot provide as large a "signal-to-
c1utter" ratio as the filter designed for a precisely known target object.
Indeed, for each specific target object with spectrum a Vx'
/y) the
11. TARGET LOCATIng IN CLUTTER 477

following inequality for "signal-to-clutter ratio" SCR for the filter of Eq.
11.3.9 holds:

(11.3.11)

The right hand part of the inequality is the signal-to-clutter ratio for the
optimal filter, exactly matched to the target object.
478 Chapter 11

11.4 LOCALlZATION METHODS FOR SPATIALLY

INHOMOGENEOUS CRITERIA

Let us return back to the general formula of Eq. 11.1.4 that accounts for
possible inhomogeneity of the localization criterion by means of weight
coefficients {W.}. Depending on the implementation constraint, one of three
ways to attain the minimum of p" may be suggested in this case.

8. Re-adjustable localization device with Jragment-wise optimal

adaptive correlator. Given nonnegative weights {W.}, the minimum of p"
is attained at the minima of integrals

P.". = Jp(bo>dboJIw.(XO'Yo(AV fh.(b/(XO,Yo)}ib)dxodyo . (11.4.1)

bg

~ ~ ~

This implies that the filter should be re-adjustable for each s-th fragment.
Frequency responses of fragment-wise optimal filters are determined as it
was shown in Sect. 11.2 for the homogeneous criterion on the basis of
measurements of spectra of individual fragments.
According to Eq. 11.1.4, the fragments are assumed not to overlap.
However, from the very sense of Eq. 11.1.4, it is obvious that it gives rise to
a sliding window processing. In the sliding window processing, 10calization
is carried out by applying to image fragments within the sliding window at
each window position the optimal adaptive correlator that is designed upon
estimates of local power spectra within the window. Then the device for
locating signal maximum analyzes obtained output image and generates
coordinates of the highest correlation peak. We will refer to sliding window
optimal adaptive correlators as to loeal adaptive eorrelators.
Experimental verification of the local adaptive correlators confirms that
they substantially outperform global optimal adaptive correlators in terms of
signal-to-cluster ration they provide. Fig. 11-5 illustrates application of the
local adaptive correlator to localization of a small fragment of one of the
stereoscopic images in the second image. Size of the fragment is 8x8 pixels,
image size is 128x256 pixels, size of the sliding window is 32x32 pixels.
Graphs on the figure illustrate difference between filter response to the target
and that to the rest of the image. The same filter applied globally rather than
locally fails to properly localize the fragment (compare with the example
shown in Fig. 11-3, when global optimal adaptive correlator provides
approximately the same signal-to-clatter ration for a target fragment of
16x16 pixels).
 11. TARGET LOCATlng IN CLUTTER 479

In conclusion note the similarity between local adaptive correlator and

local adaptive filters for image denoising described in Sect.8.3.

RHUIl oIlho Ioeol;zalöo<\ (m."'.., wUh • trosa)

OuIQUI 011110 OAC (column) 0"","1 ollhe OAC (row)

~ ~--------~----------~
~ r---~----~------------n ~r----,~-----;---;------ ....-i
ror---------~r.-.__1----__H 201---~---t---'--:----'----+-t-i

Figure 11-5. Localization of a small (8x8 pixels, highlighted) fragment of the right of two
stereoscopie images on the left image by the local adaptive correlator.

b. Non-re-adjustable localization device. When there is no possibility of

implementing the local adaptive localization device with fragment-wise or
sliding processing , the alternative is to design a localization device adjusted
to the power spectra of image segments averaged over the set of segments
with weights {Ws}, Indeed, it follows from Eq.(11.1.4), that

~ = -1P(b.ldb, t. w, [! J
w, (x" y,){ AV.. h, (b t(x"y, )~b r'dy, =
Jp(ho)dhoAV Jhs(h~b, bg (11.4.2)
bo

where h.(b) is a histogram averaged over both {WJ and {ws(xo,Yo)}'

480 Chapter 11

Now one can, by analogy with Eqs.l1.3.9, conclude that the optimal filter
may be designed upon averaged power spectra of background components of
image segments as

(11.4.4)

where AVs denotes averaging power spectra p,!;){tx,/yr of background

components of image fragments over the image area:

Thus, the optimal filter frequency response in this case depends on the
weights {Ws} and is determined by the averaged power spectrum of all
segments. Of course, such an "averaged" filter may not be as efficient for
individual image fragments as the filter adapted to these fragments but it is
definitely less expensive for computation and hardware implementation.
One can also notice that if a set of images is given for which the filter should
be optimized, the averaging should be extended to this set. In the limit, it
reduces to the estimation of the image statistical power spectrum, and we
arrive at the optimal filter of Eq. 10.4.2 for additive non-white Gaussian
noise.
The above reasoning mayaIso be applied to pattern recognition tasks.
When it is required to recognize an image from a given set of images. It
follows from Eq. (11.4.4) that, prior to recognition, images in the set should
be normalized and the normalization can be implemented in the Fourier
domain by the filter:

(11.4.6)

where k is image index and N is the number of images in the set. The
recognition can then be carried out by filtering the normalized images with
filters
 11. TARGET LOCATIng IN CLUTTER 481

(11.4.7)

and by subsequent determining the filter that provides the highest output.
c. Image homogenization. In eases when non re-adjustable loealization
deviee does not provide admissible loealization reliability while one eannot
afford the eomputational expenses for the implementation of the loeal
adaptive eorrelator, an intermediate solution, an image preproeessing that
improves image homogeneity may be used (~IJI~ After the preproeessing,
optimal adaptive eorrelator should be built for the preproeessed image. We
will refer to such a preproeessing as to image homogenization.
The idea of the image homogenization arises from the following
reasoning. In terms of the design of the optimal adaptive eorrelator, image
may be regarded homogeneous if speetra of all image fragments involved in
the averaging proeedure in Eq. (11.4.5) are identieal. While this is, in
general, not the ease, one ean attempt, before applying the optimal adaptive
eorrelator, to reduee the diversity of image loeal speetra by means of an
appropriate image preproeessing.
One of the eomputation-wise simplest preproeessing homogenization
methods is the preproeessing that standardizes image loeal mean and loeal
varianee. In diserete signal representation, it is defined for image fragments
of (2N) + lX2N2 + 1) pixels as

(11.4.8)

where (k,l) are running coordinates of the window eentral pixel, {bk,/} are
image sampIes,

(11.4.9)

is image loeal mean over the window and

(11.4.10)

is its loeal varianee.

482 Chapter 11

Such an image homogenization procedure equalizes average of local

power spectra over all frequencies rather then entire spectra and may be
regarded as a zero order approximation to making spectra identical on each
particular frequency. As it was described in Sect. 7.1, computation of image
local mean and variance may be very efficiently carried out recursively such
that the computational complexity may not depend on the window size.
Selection of the window size is governed by the image inhomogeneity. A
good practical recommendation is to select the window size commensurable
with the size of the target object. Localization with the use of the
homogenization preprocessing is illustrated in Fig. 11-6 on an example of
localization of corresponding points on the same stereoscopic images as
those used in Figs. 11-3 and 11-5. Comparing images in these figures, one
can see that the homogenization may visually be interpreted as an
enhancement of local contrast of image details. Note that such a
preprocessing may be also used for image enhancement aimed at easing
image visual analysis (see Sect. 8.6.4). Its relation to the problem of
localization of objects in images provides a good explanation why it does
help to visually analyze images.

Result of the localization (rnarked with a cross) Target (higbligbted)

Output of the rnatched fdter (row) Output of tbe rnatc:bed fdter (column)

0.5 0.5 ... .

o o

-0.5
-0.5 ····· ····i···········i········+·······+········· ~ ........!...
50 100 150 200 250 20 40 60 80 100 120

Figure 11-6. Illustration of improvement of the localization with the use of image
"homogenization"
 11. TARGET LOCAT1ng IN CLUTTER 483

11.5 OBJECT LOCALIZATION AND IMAGE BLUR

In this section we analyze how such cornmon image distortion as blur

affects the localization relaibility. We will assume that the optimal adaptive
correlator ofEq. 11.2.15 is employed in the localization device.
As it was described in Sect. 8.2.3, image blur can usually be described in
terms of distortion of image spectra in imaging systems. Yet another image
distortion factor that has to be taken into account is imaging system noise.
Let frequency response of the imaging system be HlmsVx'/y) and let the
noise be additive with spectral density N{/x' /y). Then, if the target object
has spectrum u{/x'/y)' its spectrum at the output of the imaging system
will be equal to u{tx,/y)n;",.{tx'/Y)' The image background component

spectrum p.!!.O){/x,/yf will also be modified by the imaging system to

become p.!!.O){tx,/yf/H;ms{tx,/yf + N{tx,/J. In this estimate we assume

averaging of the background component spectrum over an ensemble of its
different possible realizations generated by the noise. The optimal adaptive
correlator should then have the following frequency response:

(11.5.1)

In order to give to this equation a clear physical interpretation, modify it

in the following way:

H~;!){tx,/J=
u*{/x,/J H;:".{tx'/y)x!!·O){tx,/yf =
p.!!.O){tx,/yf/Hlms{tx,/yf p.!!.O){/x,/yf/H;.... {tx,/yf + N{/x'/y)

_ u*{tx,/J [ 1 /Hlms{/x,/yfp.!!·O){/x,/yf]
-i".(0.0)(1" I~l H;•..(/x,/Ji".(o.O)(1" I"~l/H (I" l~l+N (I" 1")'
~bg J x' y~ ~bg Jx'Jy~ Im. Jx' y~ im. Jx'Jy

(11.5.2)

The first factor in this formula

484 Chapter 11

H (~ ~)_ a:{t",tJ (11.5.3)

opt J,,'Jy - L.,.(O,O)( ~2
r"bg t",ty~

is obviously the optimal adaptive filter designed for the undistorted image.
As for the second factor,

(11.5.4)

it may be regarded as the frequency response of an image deblurring filter.

As one can see, it is an empirical Wiener deblurring filter introduced in Sect.
8.2.3. This means, that the optimal adaptive correlator defined by Eq. 11.5.2
may be treated as a device that performs a deblurring operation and then
optimal adaptive correlation for the deblurred image.
It will be instructive now to estimate "signal-to-clutter" ratio that this
filter can provide. By analogy with Eqs. 11.2.12 and 13, one can obtain in
this case that

(11.5.5)

where SCRld is signal-to-clutter ratio for the nonblurred image. One can
evaluate from Eq. (11.5.5) how much image blur worsen the localization
reliability. In particular, Eq. (11.5.5) shows that so long the noise level in the
imaging system is not very high, image blur may have marginal effect on the
performance of the optimal filter.
 11. TARGET LOCATlng IN CLUTTER 485

11.6 OBJECT LOCALIZATION AND EDGE

DETECTION. SELECTION OF REFERENCE
OBJECTS FOR TARGET TRACKING

It is widely believed in the image processing community that the

information conveyed by images is contained mostly in image high
frequency components and in edges. The theory of the optimal adaptive
correlator provides a rational explanation of this belief. Represent, for
instance, Eq. (11.2.13) for the optimal adaptive correlator in the following
way:

In this representation, the optimal adaptive correlator is regarded as

consisting of two filters in cascade. The filter represented by the second
factor in Eq. (11.6.1), may be regarded as an analog of the whitening filter
introduced in Sect. 10.4 for optimal target localization in correlated Gaussian
noise. This filter being applied to the input image makes the image
background component spectrum almost uniform. Practically, the same may
be said with respect to the input image spectrum for all reasonable estimates
of image background component power spectrum from the power spectrum
of the entire input image. The filter represented by the first factor in Eq.
11.6.1 is, obviously, nothing but a matched filter for the target object,
modified by the same whitening operator.
This implies two conclusions. First of all, power spectra of images
usually (though not always) tend to decay on high frequencies. Therefore
spectrum whitening usually results in emphasizing high frequency image
components with respect to its low frequency components. It is in this sense
one can tell that high frequency image components are more important for
image object recognition. Second, if one visually compares image before and
after whitening such as, for instance, image shown in Figs. 11-7 and 8, one
can see that image whitening results in what is visually interpreted as edge
enhancement.
An important feature of optimal adaptive correlators is their adaptivity.
Their frequency responses are determined by power spectra of the input
images. Therefore the whitening operation is, in principle, also adaptive. In
general, the whitening tends to suppress those image frequency components
that have high energy and that are, therefore, responsible for features
common to the majority of objects represented in image. Low energy
frequency components that are responsible for "uncommon", or rare and to
486 Chapter 11

objects and object features are, on contrary, emphasized. In asense, one can
say that whitening is an operation that automaticaIly enhances dissimilarities
and suppresses similarities of objects in images. This is weIl illustrated by
whitening of a test image of geometrical figures and printed characters
shown in Fig. 11-8 and of a test texture image shown in Fig. 11-9. In Fig.
11-8 one can see that the whitening does enhance edges but the enhancement
is very selective. Vertical and horizontal edges that are common to aIl
figures and characters are enhanced much less then circumferences, slanted
edges and corners in geometrical figures and characters and those are also
enhanced very selectively according to their rate of occurrences in the
image. Whitening of texture image in Fig. 11-9 mayaiso be interpreted as
edge enhancement but the nature of this enhancement is substantiaIly
different from that in Figs. 11-7 and 8. In this example, the whitening
automaticaIly detects boundary between different textures that differ in their
spatial statistical properties rather then in their gray level distribution.

Initial image "Whitened" ima~e

Figure 11-7. Image whitening as edge detection

Initial test image Whitened image and its magnified fragments

Figure 11-8. Image whitening: edges and corners in geometrical figures

 11. TARGET LOCATlng IN CLUTTER 487

Test texture image Whitened image

Figure 11-9. Whitening ofa test texture image

It is very instructive to analyze impulse response of the whitening filter.

Diagrams on Fig. 11-9 show several most valuable central sampies of 1-D
sections of impulse responses of the whitening filters for the air photo graph
and for the test image of geometrical figures shown in Figs. 11-7 and 8.

0.8 0.8
~ .
0.6 0.6

0.4 0.4
. - 0.2
0.2

~-~~I · I~~~-~
-0.2 L....._ _ _ __ ~_---' -0.2 L.....--'_--'-~~_ __ ~

Air photograph Geometrical figures

Figure 11-10. 1-D sections of impulse responses of whitening filters for two images of
Figs. 11-7 and 8

One can see that those impulse responses are not identical though they
are rather similar. The remarkable fact is that the dissimilarity between the
whitening filter impulse responses is not commensurable with the
dissimilarity between the two images. This may be attributed to the
statistical nature of the image power spectra measured for the entire image
488 Chapter 11

with many objects in it. One can also note that whitening filter impulse
responses resemble very much the impulse response of the filter that
computes image spatial Laplacian defined by the matrix:

(11.6.2)

where CI and c2 are non-negative constants such that CI < c2 and

CI + c 2 = 0.25. Most frequently settings CI = 0 and c 2 = 0.25 are used.
This observation suggests an explanation and a justification to the
common belief in the importance of the Laplacian operator in image
processing (see, for instance, [6]).
One can get an additional insight into the importance of different
frequency bands in images from evaluating the potential "signal-to-clutter'
ratio at the output of the optimal adaptive correlator. The "signal-to-clutter"
ratio SCR is a natural figure of merit of target objects that determines the
potential reliability oftheir localization. One can obtain from Eqs. (11.2.13-
11.2.15) that for the optimal adaptive correlator

(11.6.3)

It follows from this formula that the signal-to-clutter ratio at the output ofthe
optimal adaptive correlator is equal to the energy of the whitened spectrum
of the target object. Therefore the higher the energy of the whitened
spectrum, the higher is the signal-to-clutter ratio that one can expect at the
optimal adaptive correlator output for this object. It is this relationship that
quantitatively measures the importance of different signal frequency
components.
This relationship provides also a useful quantitative measure for the
selection of target objects in cases when target objects have to be chosen
from the image. This is the case, for example, in image registration and
matching with a map. Similar problems occur in stereo image analysis, in
robot vision navigation, in multi modal medical imaging. In such
application, the question is how to make the choice of the target objects to
the best advantage. As it follows from Eq. (11.6.2.) those image fragments
will be the best target objects that have maximal energy of their "whitened"
power spectrum. Therefore, the reference objects with intensive high
 11. TARGET LOCATlng IN CLUTTER 489

frequency components, that is, image fragments that are visually interpreted
as containing the most intensive edges or texture, will be the best candidates.
This conclusion is illustrated in Fig. II-11 where the "goodness" of the
fragments of 7x7 pixels in terms of the SCR-factor is represented by the
pixel gray levels for test images of Figs. 11-8 and 9. One can see from these
images that edge rich areas are indeed most appropriate potential candidates
for reference object. However, the remarkable observation is that, for
rectangles in the image of geometrical figure and for the image of periodical
textures, corners have much higher "goodness". This result is very natural.
Edges in all rectangles are similar and cannot be regarded as their specific
features. For the texture image, corners of areas of textures with different
orientation are certainly the more specific objects.

Figure 11-11. Potential reference objects for test images of Figs. 11-8 and 9: object
"goodness" is displayed proportional to pixel gray levels
490 Chapter 11

11.7 OPTIMAL ADAPTIVE CORRELATOR AND

OPTICAL CORRELATORS

11.7.1 Optical correlators with improved discrimination capability

After the introduetion of holographie matehed spatial filters and optieal

eorrelators ([7]) it was reeognized very soon that the ability of matehed
filters to diseriminate effeetively between an objeet of one class and objeets
belonging to other classes is not nearly so good as its ability to eombat
additive Gaussian noise. Aeeordingly, attempts were undertaken to find
filters with better diseriminatory eapability.
One of the oldest ideas is that by S.Lowenthal and Y.Belvaux ([8]) of
preproeessing the pieture as weH as the referenee objeet by an appropriate
differential operator before they are eorrelated. This ean be aeeomplished by
spatial filtering with eoherent light; for example, by using opaque stops or
slits, eentered on the optieal axis, or by introdueing into the eoherent optieal
eorrelator an additional spatial filter.
Another simple method for inereasing diserimination by differentiation is
to use the nonlinearity of the medium on whieh the holographie matehed
filter is reeorded. It was noted that overexposing the speetral information
from the target material of whieh the filter is made ean result in high pass
filtering similar to the spatial differentiation of ([9,10]). More reeently it was
noted that Joint Transform Correlators appear to have better diserimination
eapability than do the matehed filters if the nonlinearity of the medium used
in their Fourier plane to reeord the joint speetrum is properly chosen.
Fabrication of the matehed filter requires, in general, recording both
amplitude and phase information. This is not a trivial proeess that requires
using special reeording media and means, espeeiaHy, if the filter is to be a
synthetie one, generated by a computer (some assoeiated problems are
diseussed in Seet. 13.3). In order to simplify fabrication of the filter J.L.
Homer and J.R. Leger proposed using, instead of the matehed filter, a
phase-only filter (POF), a filter with eonstant amplitude transmittanee and
phase transmittanee equal to that of the matehed filter ([11,12]). If the
frequeney response of the matehed filter is a.. Vx , /y ), frequeney response
of the phase-only filter is

(11.7.1)
11. TARGET LOCATIng IN CLUTTER 491

An additional simplification is the so called binary phase-only filter (BPOF)

[11,12] in which only the signs of the real and imaginary parts of the
matched filter complex transmittance are recorded. Having been introduced
mainly to avoid the need for complex filters and to increase the filter
efficiency in using light energy in the optical correlators, POF and BPOF
appeared to have better discrimination capability than the conventional
matched filter. This fact along with their simplicity of implementation
contributed greatly to the popularity ofthe POF.
Many different improvements of the POF have been proposed. Here are
some of them, to name a few:
amplitude compensated matched filters (ACMF, [13]):

(11.7.2)

amplitude-modulated POF (AMPOF, [14])

(11.7.3)

where E 2 is a small constant introduced to prevent dividing by zero;

POF with an improved signal-to-noise ratio (POFISNR, [15]):

POFISNR(/)= { ~(
a' Vx ,/J
)2' or
fi (I
x'J y E
1") F
, (11.7.4)
f'"" /x'/y~
0, otherwise

where F is an area in Fourier domain chosen by some optimization

procedure to improve signal-to-noise ratio;
A family oftemary matched filters TMF ([16]), a natural generalizations
ofthe BPOF, in which the real and imaginary parts ofthe matched filter
transmittance are quantized to 3 levels (1, -1, 0) instead of two, and the
location of the areas where transmittance of the filter must be zeroed, is
chosen by an optimization procedure designed to improve signal-to-
noise ratio;
Optimal BPOF (OPOF) [17]:

(11.7.5)
492 Chapter 11

where binary function e(/" , /y) that takes values 0 and 1, and constants
{«PI ,4>2} have to be chosen to optimize the filter figures ofmerits.
"Minimum Average Correlation Energy" filters (MACE- filters, [18]).
They maximize the ratio of the squared peak value of the correlation
function to the average correlation plane energy. The MACE-filters are,
in principle, designed to identify (or detect) multiple targets (in the
presence of virtually different types of uncertainty in their apriori
description), but for precisely known objects they coincide with the
ACMF-filters.
Entropy Optimized Filters (EOF [19]) that provide a strong and narrow
peak for a match between the input and filter function as contrasted with
a uniform signal distribution for any pattern to be rejected. It was shown
by computer simulation, that the discriminatory power of EOF is much
better than that of POF and is comparable with that of MACE-filters. It
was also noted, that a dominant feature of the EOF, observed
experimentally, is substantial enhancement of the high frequency
components of images. Another important feature of the EOF is its
adaptivity, because the filter optimality criterion takes into account the
patterns to be rejected.
Phase-only correlators (POC[20]):

a:(t",fJ
", y-IßV",/y~ ~v",/yr
POC(f /)_ 1 (11.7.6)

where ß(t" '/y) is spectrum of the image in which the target object has
to be detected.
As one can see, the design of only two latter filters takes, in one or
another way, into ac count objects that have to be discriminated from the
target object by the filter. All other filters are optimized only in terms of the
sharpness of the peak of their response to the target object. Obviously, the
latter does not necessarily seeures the high discrimination capability of the
filters with respect to non-target objects. Nevertheless, all these filters may
be regarded as non-adaptive approximates to the optimal adaptive correlator
defined by Eq.(11.2.15) and its implementations described in Sect. 11.2.2.
This explains their improved discrimination capability as compared to the
matched filter.
For instance, frequency response of the optimal adaptive correlator may
be approximated as

(11.7.7)
 11. TARGET LOCAT1ng IN CLUTTER 493

when power spectrum of the input image is used as a zero order

approximation for the power spectrum of its background component. If the
input image contains only the target object without any non-target objects,
the second filter in the formula (11.7.7) will just be the phase-only-filter, and
the total filter (11. 7.7) will simply be the amplitude-compensated -phase-
only filter, as weIl as a MACE-filter designed for a single object. Of course,
at the filter output we shall have a delta-function. Furthermore, in pattern
recognition the main danger of false recognition is associated with non-
target objects most nearly similar to the given target object. Hence, in this
case, the phase-only-filter represents a more or less good approximation to
the second filter in (11.7.7), and the other filters mentioned above
approximate the optimal filter itself. But it is precisely the extent of the
dissimilarity between spectra of target object and non-target objects that is
the most important for discriminating objects by optimal adaptive
correlators. Optimal adaptive correlators take advantage of these
dissimilarities.
As it was shown in Sect. 11.6, the inherent part of the optimal adaptive
correlation is the whitening operation. Owing to the fact that usually (hut not
always!) image power spectra decay at higher spatial frequencies, whitening
results in enhancement of high frequencies. This why the above mentioned
recommendations about enhancement of high frequencies improve the object
detectibility. But, in distinction to these ad hoc recommendations, Eqs.
11.6.1 and 11.7.7 explicitly indicate to what degree and in which specific
way this enhancement must be carried out for each specific image with its
specific set ofnon-target objects.
Above mentioned improvements in discrimination capability of Joint
Transform Correlators achieved by proper selection of nonlinearity of the
medium used for recording joint spectrum mayaiso be explained in a similar
way as it will be shown in the next section.

11.7.2 Computer and optical implementations ofthe optimal adaptive

correlator

Computer implementation of the optimal adaptive correlator is most

naturally based on convolution in Fourier domain because its design is based
on measuring image power spectra as it is described by Eqs. 11.2.15, 11.2.16
and 11.2. 17. Flow diagram of the corresponding computer algorithm is
shown in Fig. 11-11. In local adaptive correlation in sliding window, this
algorithm is applied locally at every window position, spectra of image
fragments within the filter window are computed recursively using the
algorithm described in Sect. 5.4. Note the similarity between local adaptive
correlation and local adaptive image restoration described in Sect. 8.3.
494 Chapter 11

Correlator' s
output
Filter formation according
to Eqs. 11.2.15 and 11.2.18

Target object image a(x)

Figure 11-12. Flow diagram of optimal adaptive correlation

Optimal adaptive correlators mayaiso be implemented in nonlinear

optical and electro-optical correlators. Nonlinear optical correlators are
optical correlators with a nonlinearity in their Fourier plane. Two types of
nonlinear optical correlators are most feasible for the implementation of
optimal adaptive correlators: optical correlators with k-th low nonlinearity
([21]) and Joint Transform correlators ([22]).
Consider a 4-F coherent optical correlators with a non-linear light
sensitive transparent medium installed in its Fourier plane (Fig. 11-12, a).
Let's assume that transparency T ofthe medium is inversely proportional to
k-th power of the intensity ] of the incident light:

1
Toc-
]k . (11.7.8)

We ill refer to this type of the nonlinearity as to k-th law nonlinearity. The
correlator of Fig. 11-12 with a matched filter a:
(Ix, I y ) installed in its
Fourier plane after the nonlinear medium will implement input image
filtering with frequency response

(11.7.9)
 11. TARGET LOCATIng IN CLUTTER 495

because light intensity in the Fourier plane ofthe correlator is proportional to

power spectrum IßVx ,/y ) 2 of the input image. Selecting nonlinear medium
with the nonlinearity index k ~ 1 and placing it in the Fourier plane slightly
out of focus or/and using low resolution nonlinear medium, one can
implement the optimal adaptive correlator of Eq. 11.2.15 with estimation of
the background image component power spectrum described by Eqs.
11.2.23, 24.

Matchcd IiIter

,
,,
,
Parallel ; ,'
laser light ~ __ E __
bea m ............
",
" ,

Input Correlation
I-st Fourier Fourier 2-nd Fourier
image plane
lens plane lens

a)

Input image Correlation output

-=·,~---r~-----
, I
, ' .... I

,,
" ........... I
I "',I .......
I
I " , I ........
I " I

I
I " ,
,
I
I

b)

Figure 11-13. Schematic diagram of optical correlators with a nonlinearity in the

Fourier plane: a) - 4-F nonlinear optical correlator; b) Nonlinear optical correlator
with a parabolic mirror

Important practical limitation of such an implementation is high dynamic

range of image power spectra which requires the corresponding high
dynamic range of the non linear medium. This requirement is substantially
weakened for the optical correlator with a parabolic mirror as Fourier
496 Chapter 11

transformer shown in Fig. 11.-12, b). In this correlator, light passes through
the nonlinear medium twice which means that the required dynamic range of
the nonlinear medium is roughly a square root of that required for the
medium in 4-F correlators.
Computer simulation of nonlinear optical correlator with k-th law
nonlinearity and its comparison with matched filter, phase-only filter and
phase-only correlators demonstrated that ([21])
Optimal value of the nonlinearity index k is c10se to 1.
Input image power spectrum moderate smoothing for the
implementation of the correlator significantly improves the
discrimination capability of the correlator evaluated in terms of
signal-to-c1utter ratio.
k-th law nonlinear correlator substantially outperforms phase-only
correlator, phase-only and filter matched filter in discriminating
target objects against non-target objects. In the experiments, signal-
to-c1utter ratio for these four types of correlators related, by the order
of magnitude, as 7:3:2: 1. At the same time, two to five times gap
was observed between signal-to-c1utter ratio for the optimized k-th
law nonlinear correlator and the ideal optimal adaptive correlator
designed, in the simulation, for the exactly known power spectrum of
the background image component.
A trade-off exists between the optimal nonlinearity index k and the
degree of the dynamic range limitation of the nonlinear medium; a
higher degree of limitation requires a higher value of k.
Combination of input image power spectrum smoothing, high
nonlinearity index k and a high degree of the dynamic range
limitation promises considerable improvement in the nonlinear
correlator's light efficiency while preserving their high
discrimination capability.
Joint Transform correlators offer yet another possibility for implementing
optimal adaptive correlator. In nonlinear JTCs, input image and target object
joint spectrum is non-linearly transformed in an electronic amplifier before
modulating the output spatiallight modulator (Fig. 11-14). Ifthe amplifier is
the logarithmic one, signal recorded on the spatial light modulator is
proportional to

Inllß (Ix ,/J+a(/x ,/Jexp(i2I xf J= t

InHIß(Ix,/yf +f(Ix,/yf J+
ß· (Ix'/y )x(/x'/y )exp(i2/xx)+ ß(Ix'/y)x· (Ix'/y )exp(- i2/xx)} =
InlIß(Ix,/yf +f(Ix,/yf J+
 11. TARGET LOCATIng IN CLUTTER 497

(11.7.10)

where, as in Eq. 10.2.12, ß{tx,f') is spectrum of the input image,

a (tx' fy) is spectrum of the target object, x is displacement between input
image and target object template in the correlator's input plane, and where a
natural assumption that

(11.7.11)

is made on the base that the size of the target object is much smaller then the
input image size.

Fourier Joint spectrum

Input Input plane lens plane
image

Target
object

Nonlinear
amplifier

Nonlinearly Fourier Output

transformed Jens correlation plane
joint spectrum

Figure 11-14. Schematic diagram ofthe Joint Transform correlator with a nonlinear
transformation of the joint spectrum

As one can see, the last term in Eq. 11. 7.10 approximates the output signal of
the optimal adaptive correlator in which power spectrum of the background
498 Chapter 11

image component is estimated as it is described by Eq. 11.2.19 for the

additive model of target objects and input images.
Logarithmic nonlinearity is not the only option for nonlinear joint
transform correlators. Nonlinearity optimization in nonlinear joint transform
correlators was addressed in Ref. [22]. It was shown there that
different types of nonlinear transformation of the joint spectrum may be
used for substantial improvement of the correlator's discrimination
capability, provided an appropriate choice of the nonlinearity
parameters: logarithmic transformation, the k-th law transformation, with
k < 1 , in combination with limitation of the dynamic range and
thresholding the joint spectrum
A moderate blur of the joint spectrum before its nonlinear transformation
is admissible and may even improve the correlator's discrimination
capability.
All types of the nonlinear joint transform correlators are tolerant to
reasonable deviations of nonlinearity parameters from their optimal
values.
 11. TARGET LOCAT1ng IN CLUTTER 499

11.8 TARGET LOCATING IN COLOR AND MULTI

COMPONENT IMAGES

11.8.1 Theoretical framework

The theory of the optimal adaptive correlator may be extended to the

problem of reliable target localization in color and multi-component images
with a localization device composed of a multi component linear filter, a
summation unit and a unit that determines coordinates of the highest signal
peak at its input (Fig. 11-15)

Input Device for

image localization of
Multi-component
components linear filter Adder signal
maximum

Object's coordinates

Figure 11-15. Schematic diagram of the localization device for multi-component images

We will consider here only the problem of localization of an exactly

known object for spatially homogeneous criterion. Following the reasoning
that resulted in Eq. 11.2.13 for the filter that maximizes signal-to-clutter
ratio for single component images, one can conclude that, for multi-
component images, multi-component filter will be optimal that maximizes
signal-to-clutter ratio at the output of the summation unit:

L L La (tx ,I y ' Im )n{tx ,Iy ' Im )

argmax S~= h~~ ,

~
H(Jx ,Iy ,I.,

{ AVbK L LILa i;,O){tx' I Im )n{tx' I y ' Im 2}1/2

Ix I y Im
y'
1
(Eq.11.8.1 )
500 Chapter 11

where (I", /y) Im is the frequency index

are indices of spatial frequencies,
ofthe component-wise Discrete Fourier Transform, a!!,O)(J,,'/y'/m) is 3-D
spectrum of the target object, a!!,O)(f,,'/y,/J is 3-D spectrum of the
background c1utter (note that here we consider discrete Fourier spectra),
horizontal bar denotes average over unknown coordinates of the target

r
object. Reformulate Eq.ll.8.1 in terms of power spectrum of the c1utter
~!!,O)(/",/y,fm that may be estimated from the observed image. To this
goal, one can rely on the inequality:

It dV.,f"f.l· HV.,f"f.f
V s
r r
(Eq. 11.8.2)
ML~ c,(J,,'/y,fm ·IH(J,,'/y'/m
I ..

where M is the number of image components, which implies that

(Eq. 11.8.3)

Right part of Eq. 11.8.3 defines the lower bound of SCR for any
spectrum of c1utter and any filter frequency response. It follows now from
Cauchy-Schwarz-Buniakowsky inequality that choosing

(Eq. 11.8.4)

guaranties that this lower bound is made the highest possible:

(Eq. 11.8.5)
 11. TARGET LOCATIng IN CLUTTER 501

Dropping out averaging AVbg over background clutter ensemble, obtain

frequency response of the optimal adaptive multi-component correlator:

(11.8.6)

For the implementation of the optimal adaptive multi-component correlator,

IßVx' I,., Im rof the image with a spatial and

one can assume, similarly to the case of single component images,
smoothing power spectrum
component-wise spectral window:

H (J f f. )= CI. *Vx,I,.,/J
Opl x' ,.' '" Iß(Ix ,1,.,1... )2~ • w (Ix,I,.,I. . ) , (11.8.7)

where W(tx,f,.,f"') is a 3-D spectral window function and symbol •

denotes the convolution operation.
In the conc1usion note that all solutions that were obtained in Sects. 11.3
and 11.4 for inexactly known target objects and for spatially inhomogeneous
criterion may be extended to the multi-component case in a straightforward
way.

11.8.2 Separable component-wise implementations ofthe optimal

adaptive multi-component correlator ([23])

3-D image processing makes serious demands to the computing resources

in digital image processing. Electro-optical implementation is also possible
only in a form of a separable, component-wise processing. Therefore we will
consider the implementation that is schematically represented in Fig. 11-16.
It assumes separable space and component-wise processing where primary
image components are first transformed before their use as inputs of multiple
channel correlators. Outputs of the channel correlators are then combined to
form a single output signal in which target is localized by locating position
of the highest peak or peaks that exceed a certain threshold.
Such a channel decomposition can be based on an assumption that power
spectrum of the background clutter ~!!.O)Vx,f,.,I",r may be
approximated as a separable function:

(11.8.9)
502 Chapter 11

loSt ebaDuel
eorrelator

2-Dd ebanDel Adder

correlator
1-----------------------
M-th ehaDDel
correlator
Target object's
CompODeut
transformation coordinates

,,
Primary image
components
Figure 11-16. A separable implementation ofthe multi-component localization device

In this case, the 10calization is implemented with separate component-wise

and spatial-wise "whitening filters"

(11.8.10)

and

(11.8.11)

respectively. The target object should also be subjected to the same

transformations. Therefore channel correlators in Fig. 11-16 should be
optimal adaptive correlators for image components transformed by
component-wise "whitening" filters defined by Eq. 11.8.10. For the
involved separated spectral factors of the background cIutter power

r
spectrum, the following empirical estimations from input image power
spectrum Iß(Jx' I y' Im may be suggested:

(11.8.12)
 11. TARGET LOCATIng IN CLUTTER 503

and

r;:){tx,/yf == IIß{tx,fy,f",f .
I ..
(11.8.13)

Flow diagram of a computer implementation of the separable optimal

adaptive multi-component correlator is shown in Fig. 11-17.

1--" - - -
Chanoel correl.tor
I-D
component-wise·
inverse DU

1-0
2-wbitening
component -wise
filter

r
2-whitenlng ruter

lIri;)(f~

1-D ...... - - - 2-D

---
component- wise i.."..,', ................. ____ .. __ ................. ,.. ," ....................... j inverse
DFl' DFr

- - ----
Channel correl.tor

Localization
of signal maximum
Input image
componenls
Target co-ordinates

Figure 11-17. Flow diagram of computer implementation of the separable optimal adaptive
multi-component correlator

Component transformation for the general case of multi component

images has not as yet been discussed in the literature. However, component
transformation in the context of object recognition in color images
represented by their RGB components has obtained considerable attention.
One of the approaches involves preprocessing of RGB components on the
base of human perception ([24]). In this case an element-wise linear
preprocessing based on the opponent-color theory proposed in Ref. [25] is
504 Chapter 11

performed over the RGß components. Another example is a projection of

color images on a certain generalized color plane suggested Ref. [26,27].
This approach is aimed also at reducing the number of components used in
correlation process and therefore at simplification of multi channel optical
setup. Of course, the option of reducing the number of channels after
transformation of the image components is open for any transformation
method. Other examples of component wise color image preprocessing
aimed at improvement of the correlator's discrimination capability are
suggested in [28] on the base of the above discussed theory and proved their
efficiency. Theyare modifications ofthe following two methods:
• Component "circular whitening":

m = 0,1,2; (11.8.14)

• Component centering (component-wise Laplacian):

_ b(m+l)mod3 + b(",+2)mod3
b (",) = b - kJ kJ • m = 0"1 2 , (11.8.15)
kJ", 2 '

where {b!~)} are sampies of color image components, m is component

index and (k,/) are coordinate indices.
As one can see, both methods involve component wise discrete
Laplacian. The first method assumes independent component wise whitening
of all pixels in 3 image components without any spectral estimation by
averaging as suggested by Eq. 11.8.12 and 13.
Component wise whitening according Eq. 11.8.10 of a color image is
illustrated in Figs. 11- 18. Fig. 11-19 shows diagram of sampies of the color
whitening filter impulse response for this particular image. One can see that
the whitening filter impulse response is not exactly the Laplacian's operator
sampies (-0.5,1,-0.5) and that it may change from image to image. But the
difference is not very substantial and experiments show that variations of the
whitening filter impulse response are also not very substantial.
 11. TARGET LOCATIng IN CLUTTER 505

Figure 11-18. Color image RGB components (upper row) and the result of component wise
whitening (bottom row)

-0. 5 2 3

Figure 11-19. Impulse response ofthe whitening filter.

LocaizItion reWI (metI!ecI with. cross); SNR=24.&201

::1
0 .'
oe

0'
o~

Figure 13-20. Localization of a target in a color image: separable optimal correlator (images
are printed in gray scale)
506 Chapter 11

Loceizatlon red (mat1<ed ....th a cr055): SNR-6.81 02 Target (N~lttIed)

..
Ct'Ioft'''&A.t&ir". 0UI1'loIA
~.
.t .,.:).nl!!l
..
C ~lW"".
. .
&I."I'U! 6' y.~1"
.
00 0 .0

I
0."
0 .4 0."
I 1III \, r l
I.'
J I

~l 'l ! ~"'II'q 1111:, ! P

o

_0" "
I
I
~ (I " I
I.
d "
r 'r ' '.
." I .1
J

100 1&0 ~ :too

Figure 11-21. Localization of a target in the single component image (Green component
ofthe image in Fig. 11-20)

Figs. 11-20 and 21 illustrate that the use of color information

substantially improves reliability of object detection and localization. Fig.
11-20 shows result of localization of a small target (highlighted in figures)
on a color image. Fig. 11-21 demonstrates the localization result obtained
using a single component (Green component in this example) of the image.
They show that "signal-to-clutter" ratios for these two cases are 24.9 versus
6.8 which is more then 3 times better for the color correlator.
 11. TARGET LOCATIng IN CLUTTER 507

References
I. L. Schwartz, "Analyse Mathematique", Hermann, Paris, 1967
2. L. Yaroslavsky, "Adaptive Nonlinear Correlatorsfor Object location and Recognition",
In:Advanced Optical Correlators for Pattern Recognition and Association, Ed.
Kazuyoshi Itoh,Research Signpost, Trivandrurn, Kerala, State, India, 1997, pp. 47-61.
3. L. Yaroslavsky, "Nonlinear Optical Co"elators with improved discrimination
Capability for Object Location and Pattern Recognition", In : Optical Patern
Recognition, Ed. F.S.T. Yu and S. Jutamulia, Cambridge University Press, 1998, 141-
170
4. R.B. B1ackman and 1.W. Tukey, The Measurement ofPower Spectra, New York: Dover,
1958
5. L. Yaroslavsky, O. Lopez-Coronado, Juan Campos, Input image "homogenization"
method for improvement the correlator's discrimination capability, Optics Letters, 15
July, 1998
6. D. Marr, Vision. A Computational Investigation into the Human Representation and
Processing ofVisual Information, W.H. Freeman and Co., N.Y. 1982
7. A. Vander Lugt, Signal Detection by Complex Spatial Filtering, IEEE Trans., IT-IO,
1964, No. 2, p. 139
8. S. Lowenthal, Y. Belvaux, Reconnaissance des Formes par Filtrage des Frequences
Spaciale~ Optica Acta, v. 14, p. 245-258, 1967
9. A. Vander Lugt, A Review ofOptical Data-Processing Techniques, Optica Acta, v. 15,
1-33, 1968
10. R. A. Binns, A. Dickinson, B. M. Watrasiewicz, Method ofIncreasing Discrimination of
Optical Filtering, Appl. Optics, v. 7,1047-1051, 1968
11. 1. L. Horner, P.D. Gianino, Phase-only Matched Filtering, Appl. Opt., v. 23, p. 812-816,
1984
12. J.L Horner, J. R. Leger, Pattern Recognition with Binary Phase-only Filters, Appl. Opt.,
v. 24, 609-611, 1985
13. G.-G. Mu, X.-M. Wang, Z.-O. Wang, Appl. Opt., v. 27, 3461,1988
14. A.A. S. Awwal, M. A. Karim, S. R. Jahan, Improved Correlation Discrimination Using
an Amplitude-modulated Phase-only Filter, Appl. Opt., v. 29, 233-236,1990
15. B. V. K. Vijaya Kumar, Z. Bahri, Phase-only Filters with Improved Signal to Noise
Ratio, Appl. Opt.,18, pp. 250-257, 1989
16. F. M. Dickey, B. V. K. Vijaya Kumar, L. A. Romero,1. M. Connelly, Complex Ternary
Matched Filters Yielding High Signal to Noise Ration, Opt. Eng. , v. 29, p. 994-1001,
1990
17. W. W. Farn, 1. W. Goodman, Optimal Binary Phase-only Matched Filters, Appl. Opt., v.
27,pp.4431-4437,1988
18. A. Mahalanobis, B. V. K. Vijaya Kumar, D. Casasent, Minimum Average Correlation
Energy Filters, Appl. Opt., v. 26, pp. 3633-3640, 1987
19. M. Fleisher, U. Mahlab, 1. Shamir, Entropy Optimized Filter for Pattern Recognition,
Appl. Opt. V. 29, pp. 2091-2098,1990
20. K. Chalasinska-Macukow, "Generalized Matched Spatial Filters with Optimum Light
Efficiency", in Optical Processing and Computing. H. H. Arsenault, T. Szoplik and B.
Macukow, Ed. (Academic Press, Boston, 1989), p. 31-45.
21. L.P. Yaroslavsky, "Optical Correlators with (-k)th Law Nonlinearity: Optimal and
Suboptimal Solutions", Applied Optics, 34, pp. 3924-3932 (1995)
22. L. Yaroslavsky, E. Marom, Nonlinearity Optimization in Nonlinear Joint Transform
Correlators, Applied Optics, vol. 36, No. 20,10 July, 1997, pp. 4816-4822
23. L. Yaroslavsky, Optimal Target Location in Color and Multi-component Images, Asian
Journal ofPhysics, Vo18, No 3,1999, pp. 100-113
508 Chapter 11

24. K. Yamaba and Y.Miyake, "Color character recognition method based on human
perception," Opt. Eng. 32, 33-40 (1993).
25. S.L.Guth, "Model for color vision and light adaptation," J. Opt. Soc. Am. 8, 976-993
(1991).
26. E.Badique, Y.Komiya, N.Ohyama, J.Tsujiuchi and T.Honda, "Color image correlation,"
Optics Comm. 62, 181-186, (1987).
27. E.Badique, N.Ohyama, T.Honda and J.Tsujiuchi, "Color image correlation for
spatial/spectral recognition and increased selectivity," Optics Comm. 68, 91-96, (1988).
28. V. Kober, V. Lashin, L. Moreno, J. Campos, L. Yaroslavsky, and M.J. Yzuel, Color
component transformations for optical pattern recognition, JOSA, A/vol. 14, No.IO,
October 1997, pp.2656-2669
 Chapter 12

NONLINEAR FILTERS IN IMAGE PROCESSING

Since J.W. Tukey introduced median filters in signal processing ([1]), a

vast variety of nonlinear filters and families of nonlinear filters for image
processing has been suggested. In order to ease navigating in this ocean of
filters we will provide in this chapter a classification of the filters described
in the literature and describe some most useful filters for image denoising,
enhancement and segmentation. The classification is aimed at revealing
general common principles in the filter design and their efficient
implementation in serial computers and parallel computational networks.
In Sect. 12.1 main assumptions and fundamental notions of pixel
neighborhoods and their attributes and estimation operations that constitute
the base of the classification are introduced. Then, in Sects. 12.2-4,
respectively, listed and explained are: typical pixel and neighborhood
attributes, typical estimation operations involved in the filter design and
typical neighborhood building operations that were found in the result of
analysis of a large variety of nonlinear filters known from literature ([2-10]).
In Sect. 12.5 classification tables of the filters are provided in which filters
are arranged according to the order and the type of neighborhood they use.
Iterative, cascade and recursive implementation of filters are reviewed in
Sects. 7 and 8. Sect. 9 illustrates some new filters that naturally follow from
the classification and in Sect.10 filter implementation in parallel neuro-
morphic structures is briefly discussed.
 510 Chapter 12

12.1 CLASIFICATION PRINCIPLES

12.1.1 Main assumptions and definitions

We will assume that images are single component signals with scalar and
quantized values.
Main common properties of the filters that constitute the base of thei
classification are:
Filtering is performed within a filter window. For processing signals
and images filter window scans input image sampie by sampie.
In each position k of the window, with k being a coordinate in the
signal domain, filters generate, from input signal sampies {b~)}
within the window, an output value a for this position by means of
k

a certain estimation operation ESTM applied to a certain subset

NBH{b!k)} ofwindow sampies:

(12.1)

We will associate this estimate of the filter output with window

central sampie and will call sub set NBH {b~)} of sampies used as inputs
to estimation operations neighborhood of the window central sampie.
The neighborhood is formed on the base of window sampie attributes
(to be discussed in Sect.12.1.2). The process of forming
neighborhood may, in general, be multi stage one beginning from the
initial W-neighborhood (Wnbh) formed from all filter window
sampies through aseries of intermediate neighborhoods. Intermediate
neighborhoods may, in addition to its pixel attributes, have attributes
of their own associated with the neighborhood as a whole and
obtained through an estimation operation over the neighborhoods
formed at the previous stage.
Nonlinear filters will be specified in terms of neighborhood forming and
estimation operations they use. Such an approach enables unifled treatment
of the filters and their structurization that can be directly translated into their
algorithmic implementation on serial computers and hardware
implementation on parallel computational networks.
This concept is schematically illustrated in a flow diagram of Fig. 12-1.
Filter window sampies with their attributes form a primary "window
neighborhood" Wnbh. On the next level, this first level neighborhood
NBH 1 == Wbnh is used to form, through a number of neighborhood building
operations, a second level neighborhood NBH z . In this illustrative example,
 12. NONLINEAR FILTERS IN IMAGE PROCESSING 511

it is used for generating, by means of an estimation operation, filter output

pixel for the particular position ofthe window.

Figure 12 -1 . The concept of image nonlinear filtering

12.1.2 Typical signal attributes

Primary signal sampie attributes that determine filtering operations are

sampie (pixel) gray values and their co-ordinates. It turns out, however, that
a number of attributes other then only these primary ones are essential for
nonlinear filtering. Table 12-1 lists typical digital signal sampie attributes
that are involved in the design of non linear filters known from the literature.
As one can see from the table, these "secondary" attributes reflect features of
pixels as members of their neighborhood.
Attributes Rank and Cardinality describe statistical properties of pixels
in neighborhoods. They are interrelated and may actually be regarded as two
faces of the same quality. While Rank, as it is described in Sect. 7.1.1, is
associated with the variational row, i.e., ordered in ascending order sequence
of neighborhood pixel values, Cardinality is associated with the histogram
over the neighborhood. "Geometrical" attributes describe properties of
images as surfaces in 3-D spaces, respectively. Membership in
 512 Chapter 12

neighborhood and Spatial connectedness are binary attributes that take

values 0 and 1 and specify topological relationship between signal sampie
and a neighborhood. Neighborhood elements are regarded spatially
connected if one can connect them by a line that passes through the sampIes
that all belong to the neighborhood.

Table 12-1. Typical attributes of digital signals

Attribute Definition
Primary attributes
Value ak Sampie (pixel) gra~ value
Co-ordinate k(a) Sampie (pixel) coordinate in the filter
window
Secondary attributes
Number of neighborhood elements
Cardinality H(a)=HIST(NBH,a) with the same value as that of
element a (defined for quantized
signals): H(a) = ~)(a-ak)
kENBH
Rank Ra=RANK(NBH,a) - Number of neighborhood
elements with values lower than
a
- Position ofvalue a in variational
row (ordered, in ascending
values order, sequence of the
neighborhood elements)
a
- Ra = 2:
.=0
H(v)
Geometrical COORD(NBH,R) Co-ordinate of the element with rank
attributes R (R -th rank order statistics)
GRDNT(NBH,k J SigIlal gradient in position k
CURV(NBH,a, ) Signal curvature in position k
Membership in the MEMB(NBH,a). A binary attribute that evaluates by 0
neighborhood and 1 membership of element a in the
neighborhood.
Spatial CONCTD(NBH,a) A binary attribute that evaluates by 0
connectedness and 1 spatial connectedness of
element a with other elements of the
neighborhood

12.1.3 Estimation operations

Typical estimation operations used in known nonlinear filters are listed in

Table 12-2. Two lasses of estimation operations may be distinguished: data
smoothing operations, and operations that evaluate data spread in the
neighborhood. In the filter design, selection of estimation operation is, in
 12. NONLINEAR FILTERS IN IMAGE PROCESSING 513

general, govemed by requirements of statisticalor other optimality of the

estimate. For instance, arithmetic MEAN is an optimal MAP- (Maximal A
Posteriori Probability) estimation of a location parameter of data in the
assumption that data are observations of a single value distorted by an
additive uncorrelated Gaussian random values (noise). It is also an estimate
that minimizes mean squared deviation of the estimate from the data. PROD
is an operation homomorphic to the addition involved in MEAN operation:
sum of logarithms of a set of values is logarithm of their product.

T.ablel 22
- . EstlmatlOn ooeratIons
Ooeration Denotation I Definition
SMTH (NBH): Data smoothing operations
Arithmetie and MEAN(NBH) Arithmetic mean ofsamples ofthe neighborhood
geometrie PROD(NBH) Product of sampies of the neighborhood
mean
Order statisties K_ROS(NBH) Value that occupies K-th place (has rank K) in the
variational row over the neighborhood. Special
cases:
MIN(NBH) Minimum over the neighborhood (the first term of
the variational row)
MEDN(NBH) Central element (median) ofthe variational row

MAX(NBII) Maximum over the neighborhood (the last term of

the variational row);
Histogram MODE(NBII) Value ofthe neighborhood element with the highest
cardinality:
mode
MODE(NBII)= arg max(H(NBH))
RAND(NBH) A random (pseudo-random) number taken from an ensemble with the
same gray level distribution density as that of elements of the
neilZhborhood
SPRD(NBH): Operations that evaluate spread of data within the neighborhood
STDEV(NBH) Standard deviation over the neighborhood
IQDIST(NBH) R_ROS(NBH) - L_ROS{NBH) ,
Interquantil distance
where 1 ~ L < R ~ SIZE(NBH).
RNG(NBII) Range MAXtNBH)-MINtNBH)
SIZE(NBH) Number of elements ofthe neighborhood

ROS operations may be optimal MAP estimations for other then additive
Gaussian noise models. For instance, if neighborhood elements are
observations of a constant distorted by addition to it independent random
 514 Chapter 12

values with exponential distribution density. MEDN is known to be optimal

MAP estimation of the eonstant. It is also an estimate that pro vi des
minimum to average modulus of its deviation from the data. If additive noise
sampies have one-sided distribution and affeet not all data, MIN or MAX
might be optimal estimations. MODE ean be regarded an operation of
obtaining MAP estimation if distribution histogram is eonsidered aposteriori
distribution of signal gray level.
RAND is a "stoehastic" estimation operation. It generates an estimate
that, statistieally, is equivalent to all above "deterministie" estimates. This
quite unusual operation is illustrated in Fig. 12-2. Images on this figure
cIearly show that replaeement of image sampies with pseudo-random
numbers by RAND-operation provides a reasonably good estimate of pixel
gray levels from pixel neighborhoods. It is especially true for higher order
neighborhoods. For instanee, one ean hardly distinguish visually image (e)
from initial image (a). Differenee (e) between these two images looks almost
ehaotic which means that no substantial visual information is lost in image
(e).

a)

d) e)

Figure 12-2. Illustration ofRAND(NBH) as a data smoothing operation: a) - initial test image
with 256 quantization levels; b) - result of applying operation RAND(Wnbh) in the window
of 7 x 7 pixels; c) - result of applying operation RAND(EV-nbh) operation in the filter
window of 7x 7 pixels and E: =E; = 10 (for the definition of EV-neighborhood see
Sect. 12.1.4); d) - difference between images (a) and (d); standard deviation ofthe difference
is 26; t) - difference between images and (e); standard deviation ofthe difference is 5.6.
12. NONL1NEAR FILTERS IN IMAGE PROCESSING 515

All above-mentioned operations belong to a class of data smoothing

operations. We will use for them a common denotation SMTH. The rest of
operations generate numerical measures of neighborhood data spread. We
will use for them a common denotation SPRD. Their two modifications,
"interquantil distance" IQDIST and "range" RNG ones, are recommended
as areplacement for standard deviation for the evaluation of spread of data ..
SIZE operation computes number of sampIes in the neighborhood, when it
does not direct1y follows from the neighborhood definition.

12.1.4 Neighborhood building operations

Neighborhood building operations may be unified in two groups:

operations that generate a scalar attribute of the neighborhood as a whole
("scalar" operations) and those (vectorial) that are used in multi stage
process of forming neighborhood. The latter generate, for neighborhood
elements, a new set of elements and their attributes. This set of elements
forms a neighborhood of the next stage. "Scalar" neighborhood building
operations are basically the same as estimation operations listed in Table 12-
2. Typical vectorial operations are listed in Table 12-3.

Toble 12 -3. Vectorial neighborhood building operations

FUNC(NBH): Element wise funetional transformation of neighborhood elements
MUL T Attr(NBH): Multiplying elements of the neigbborbood by some weigbts
MULT ClNBHl weighting coefficients are defined by element co-ordinates
MULT V(NBH) weight coefficients are defined by element values
MUL T R(NBJI) weight coefficients are defined by element ranks
MULT_H(NBH) weight coefficients are defined by the cardinality of the
neiJ~hborhood elements
MUL T_G(NBH) weight coefficients are defined by certain geometrical attributes
of the neighborhood elements,
MULT_AA(NBH) weight coefficients depend on combination of attributes, for
instance, on both co-ordinates and ranks of neighborhood
MUL T CR(NBH) elements
REPL_Attr(NBH): Replieating elements ofthe neighborbood eertain number oftimes
aeeording to elements' attributes
SELECT_Attr(NBH) Attribute controlled selection of one sub-
neighborhood from a set: SELECT_A(NBH) =
Sllbnbh
MIN_Std(SllbWnbhJ, SllbWnbh2•.. ~ Standard deviation over the neighborhood as the
SllbWnbhn ) attribute
MIN_RNG(SllbWnbhJ, SllbWnbh 2• ...• Neighborhood range as the attribute
SllbWnbhn )
516 Chapter 12

r.able 12 -3 (.contd). V ectona. I nelgl

. hborh00d bUI·Id·mg operatIons
C-neighborhoods: pixel co-ordinates as attributes
SHnbh Selection of neighborhood elements
Shape-neighborhoods according to their co-ordinates. In 2-D and
multi-dimensional cases: neighborhoods of
a certain spatial shape.
V-neighborhoods: pixel values as attributes
"Epsilon- V" -neighborhood of A subset of elements with values {an} that
element ak :
satisfy inequality: ak - 8; S; an S; ak + 8; .
EVnbh(NBH;ak;8;;8; )
"K nearest by value"-neighborhood of A subset of K elements with values {a n} closest
element ak
to that of element ak .
KNVnbh(NBH;ak,K) :
Range-neighborhood: A subset of elements with values {Vk} within a
RNGnbhJ.NBH, V",,,, V1IfX)- specified range {V"'n<Vk<V'....)
R-neighborhoods: pixel ranks as attributes

ERnbh(NBH;a k;8 ;;8;) A subset of elements with ranks {Rn} that

"epsilon-R" - neighborhood satisfy inequality:
Rk - 8; S; Rn S; R k + 8; .
KNRnbh(NBH ;ak , K) - A subset of K elements with ranks closest to
"K-nearest by rank" neighborhood of that of element ak •
elementa
Qnbh(NBH, Rieft' R'lght ) Elements (order statistics) whose ranks
Quantil-neighborhood {R, }satisfY inequality
1< Rieft< R, < R right < SIZE(Wnbh)
H-nei2hborhoods: pixel cardinalities as attributes
CLnbh(NBH;a k ) - Neighborhood elements that belong to the
same cluster of the histogram over the
"Cluster" neighborhood of element a k . neighborhood as that of element a k .

FLAT(NBH)-
"Flat"-neighborhood
I
G-neighborhoods: Geometrical attributes
Neighborhood elements with values of Laplacian (or
module of gradient) lower than a certain threshold
Linear combination of elements of nei2hborhood
T(NBH) I Orthogonal transform T of neighborhood elements
DEV(NBH,a) I Differences between elements of the neighborhood and certain value a

One can distinguish the following groups of vectorial neighborhood

building operations:
functional element-wise transformations;
replication operations;
selection operations;
operations that use sampIe co-ordinates as attributes (C-neighborhoods);
operations that use sampIe gray values as attributes (V-neighborhoods);
 12. NONLINEAR FILTERS IN IMAGE PROCESSING 517

operations that use sampie ranks as attributes (R-neighborhoods)

operations that use sampie cardinalities as attributes (H-neighborhoods);
operations that use geometrical attributes (G-neighborhoods);
linear combination operations.

By functional transformations we assume applying, element-wise,

nonlinear functions such as, for instance, logarithmic one, to the
neighborhood elements. A special case of the functional transformations,
MUL T -operations, is multiplying neighborhood elements by scalar "weight"
coefficients that are selected according certain attributes (co-ordinates,
value, rank, cardinality), or according to a combination of attributes.
Replication REPL-operations can be regarded as aversion of weighting
with integer weights and are used in data sorting.
Selection SELECT_ A operations select from the neighborhood some
sub-groups of elements, or sub-neighborhoods according to certain attributes
of the sub-neighborhoods such as, for instance, standard deviation od signal
values in the sub-neighborhoods or sub-neighborhood range.
Neighborhoods with elements selected according to their coordinates in
the filter window are referred to as shape-neighborhoods as they correspond
to certain spatial shapes, such as, for instance, circle, cross, diagonal, etc.
Shape neighborhoods are key notions in morphological filters ([2]).
Signal gray value is one of the most important signal attributes. Two types
of neighborhood are based on it: EV- and KNV-neighborhoods. The former
is composed of signal sampies with gray values that deviate from the gray
value of the window central sampie not more then by certain predefined
values ~: ,E;}. The latter unify in the neighborhood certain predefined
number K of window sampies with gray values closest to that of the window
central sampie.
For ranks as signal attributes, one can introduce such analogs of EV- and
KNV-neighborhoods as ER-, KNR- neighborhoods. Quantil Q-
neighborhood is also a variant ofthe rank based R-neighborhoods.
EV-, KNV-, ER-neighborhoods are used in a number of very efficient
noise c1eaning algorithms (see Sect. 12.4). Fig. 12-3 illustrates the principle
feature controlled selection ofneighborhood elements on an example of EV-,
KNV-, ER-neighborhoods.
Cluster CL- neighborhood is an example of neighborhoods built using
pixel cardinality as the attribute. Neighborhoods built using filter window
geometrical attributes are exemplified by FLAT -neighborhood composed of
pixels with low gradient.
Neighborhoods built using linear combination of widow sampies
operations are exemplified by spectra of window sampies obtained using
orthogonal transforms such as DCT, DFT, Haar or alike and by differences
518 Chapter 12

between window sampies and certain value, such as, for instance, that of the
window central pixel.

J) •.•••• ~. • •••.•••••••••••••••••••.•

2S .•••..• ! ....... :..... .

;n • r · ••

15 ••• • •••••

- _._ .. _· · ·······r······

100 150 m :l5O

Figure-12.3. Examples offonnation ofEV-, KNV- and ER-neighborhoods: a) - input image;

b) - image fragment in a window 3lx3l pixels (highlighted in (a» with window central pixel
marked by the white square; c - histogram of the fragment with central pixel gray value
indicated by a line; d-f - results of the fragment segmentation to EV, KNV and ER -
neighborhoods (shown in white on black background).
 12. NONLlNEAR FILTERS IN IMAGE PROCESSING 519

12.2 FILTER CLASSIFICA TION TABLES

12.2.1 Transferential filters

In this section we provide, in tables 12-4 through 12-7, a list of a number

of the most nonlinear filters classified according to the number of stages in
building neighborhood they use for generating final estimation of the filter
output. We call them transferential in order to distinguish them from filters
with feedback described in Sect. 12.2.2. The list is mostly based on data
collected in Ref.[9] as weIl as in Refs. [3,4, 8, 9, 10] to which readers can
refer for detailed information regarding properties and applications of the
filters. Most of the filters are intended for cleaning additive noise in data. In
other cases we provide an additional indication of the filtering purpose.
Table 12-4 lists the simplest nonlinear filters that use one-stage NBH1
neighborhood, the primary window Wnbh-neighborhood. In particular, one
can find in the table such popular in signal and image processing filters as
moving average, median and local histogram equalization filters. We will
illustrate capabilities of these filters in Sect. 12.4.
It appears that the majority of known nonlinear filters belongs to the
family of two-stage NBH2 neighborhood filters listed in Table 12-5.
According to the type of the NBH2-neighborhood used, the filters form four
groups: MULT_A-, REPL_A-, V-, and R-neighborhood filters. In Sect.
12.4 we will compare capabilities of some of these filters with corresponding
one stage filters.
Among three-stage neighborhood filters listed in Table 12-6 one can find
two large families of filters: transform domain filters and stack filters.
Transform domain filters nonlinearly modify transform coefficients of filter
window sampies to generate filter output by means of applying to them
operation MEAN which is an implementation of the inverse transform for
the window central sampie. Two the most advanced modifications of these
filters are sliding window DCT ([8,11]) and wavelet shrinkage filters ([12]).
They are described in details in Sect. 8.3. A popular in image processing
community Local Linear Minimum Mean Square Error (LLMMSE-) filter is

r
a special case of transform domain filters in which signal squared transform
coefficients IT(Wnbh (signal spectral estimations) are replaced by their
mean values (STD(Wnbh)Y .
Stack filters belong to yet another large family of filters. They originate
from the idea of threshold decomposition of multilevel signals to binary
signals to which Boolean functions are then applied ([13]).
Four stage neighborhood filters are exemplified in Table 12-7 by a family
ofpolynomial filters and by Weighted Majority of m Values with Minimum
520 Chapter 12

Range (WMVMS-, or Shorth-) filter that implements an idea of data

smoothing by averaging over data subset that has minimal spread.

Table 12-4. One stage (Wnbh-based) filters

Silma) "smoothine" filters
Moving average filter
a = MEAN(Wnbh)
k

"Ranked order"
("percentile") filters
a = K _ ROS(Wnbh)
k

Median filter Qk = MEDN(Wnbh)

MAX-fiIters ak = MAX(Wnbh)
MIN- filters a = MIN(Wnbh)
k

Adaptive Mode Quantization filter ak = MODE(Wnbh)

Siena) "enhancement" filters
Local histogram
equalization
ak = RANK(Wnbh)
Quasi-range filter ak =QSRNG(Wnbh) =
R ROS(Wnbh) - L ROS(Wnbh)
Local variance filter ak = STDEV(Wnbh)
- Two stage (NBH]-base d) fil1 ters
Ta ble 125
FUNC(NBffl
General ak =MEAN(FUNC(NBH))
MUL T -A- neighborhood filters
RMSE optimal linear filters ak = MEAN(MULT_C(Wnbh»)
L-filters, Rank Selection filters; ak = MEAN(MULT_R(Wnbh»)
C-filters (LI-filters)
ak =MEAN(MULT_RC(Wnbh))
REPL-A - neighborhood filters
Weighted median filters ak =MEDN(REPL C(Wnbh»;
Weighted K-ROS - filters ak = K_ROS(REPL_C(Wnbh»
Morphological filters Dilation filter: ak =MAX(SHnbh)
Erosion filter ak = MIN(SHnbh)
Soft Morph. filters ak = ROS(SHnbh)
V-neiehborhood filters
K-Nearest Neighbor filter ak =MEAN(KNV(Wnbh;ak;K))
"Sigma"- filter ak = MEAN(EVnbh(Wnbh;ak;E;;E;))
Modified Trimmed Mean
filters
ak = MEAN(EV(Wnbh;MEDN(Wnbh ~E;;E;))
 12. NONLlNEAR FILTERS IN IMAGE PROCESSING 521

T.able 12-5 (Cont.

d) Two sta~e (NBd-based) fil
I ters

R-nei2hborhoods
Alpha-trimmed mean,
median
ak = MEAN(Qnbh(Wnbh,R/eft,Rr;ghl ));
a = MEDN(Qnbh(Wnbh,R,eft,Rrighl))
k

Impulse
noise
General a = MEMB(Qnbh(wnbh, Rieft' R"ght1a a
k k )· k +
filtering [1-MEMB(Qnbh(wnbh,R,eft,Rr;ghI1ak)]x
filters
SMTH(Qnbh(Wnbh, Rieft' R nghl ))
Rank a = MEMB(Qnbh(Wnbh,Rleft,R,;ghJa
k k )· ak +
II - MEMB(Qnbh(Wnbh, Rieft' Rrighl 1a
Conditioned
Median filter k) j X

MEDN(Qnbh(Wnbh, Rieft' R nght ))

Table -6. Tree stage (NB -based) filters

Transform
domain
"Soft"
thresholding
a
k = MEAN(H· T(Wnbh»)
filters H=d;aglmax~T(Wnbh)12
-cr 2IIT(Wnbh)1 2 ,O)J
"Hard"
thresholding
a k = MEAN(STEP~T(Wnbh)l-cr }. T(Wnbh»)
,

{o.x,;; °
where 0 is a filter parameter,

STEP(x)=
1,x> 0
LLMMSE-
filter a k = 1- (
A (
(0
STD Wnbh
))2 ak + ( (0
2
STD Wnbh
J 2
)y MEAN(Wnbh) ,
where cr 2 is a filter parameter
Double Window
Modified Trimmed
a = MEAN(EVnbh(Wnbh;MEDN(SHnbh~E;;E; )).
k

Mean filter
...
Stack filters
ak =
MAX(MIN(SubWnbhl~MIN(SubWnbh2~ •••,MIN(SubWnbhn)).

Table -7. Four stage (NB -based) filters

...
Polynomial ak =
filters MEAN(MULT_ C(PROD(SubWnbhl ~.•., PROD(SubWnbhn)))

WMVMS ak = MEAN(MULT_R(MIN_RNG({SubRnbh}m)}))),
- filters
where {SubRnbhf m )} are rank based sub-neighborhoods of m
elements.
522 Chapter 12

12.2.2 Iterative filtering.

An important common feature of the nonlinear filters is their local

adaptivity: the way how filter output is computed depends, in each filter
window position, on window sampie attributes. As it was already mentioned
in Sect. 8. 3, this filtering method may mathematically be explained with the
use of local criteria of processing introduced in Sect. 2.1.1. According to E~.
2.1.4, optimal processing filter is the filter that, from window sampies ~~k) J
in the window position k , generates estimates ~~k)} of the window sampies
that minimize average deviation {LOSS(a~),a~k»)} of the estimates from
sampies {a ~k) } ofa hypothetical true signal

(12.2.1)

The averaging in the criterion is two-fold: statistical averaging over an

ensemble of images for which the filter is intended and a spatial averaging.
The spatial averaging over window sampie index n is, in general, a
weighted summation carried out over a sub set of signal sampies associated
with central sampie Qk ofthe window that form its neighborhood NBH and
are defined by a locality function LOC(m;Qk). As one can see from Eq.
12.2.1, the locality function, and therefore the neighborhood, in principle,
depends on "true" value Qk of the central sampie. Therefore optimal
processing algorithm depends on signal true values that are required to
specify the locality function. For these values are not known and are the goal
of the processing, this implies that optimal estimation algorithm should, in
principle, be iterative:

(12.2.2)

where t is iteration index. In iterative filtering, filters are supposed to

converge to "true" values. Therefore, in particular applications, filters should
be selected according to their "root" signals (fixed points).
Experimental experience shows that iterative nonlinear noise cleaning
filters substantially outperform non-iterative ones. We will compare iterative
and non iterative filtering in Sect. 12.3.
 12. NONLINEAR FILTERS IN IMAGE PROCESSING 523

12.2.3 Multiple branch parallel, cascade and recursive filters

An important problem of iterative filtering is that of adjustment of

neighborhood building and estimation operations according to
changing statistics of data that takes place in course of iterations. This
may require iterative wise change of the filter parameters. One of the
possible solutions of the problem is to combine in one filter several
filters acting in parallel branches and switch between them under
control of a certain auxiliary filter that evaluates the changing
statistics.
Iterative filtering and modification of the estimation and
neighborhood building operations mayaIso be implemented in
cascade filtering when each filter in cascade operates with its own
neighborhood and estimation operation. Several examples of cascade
filters are listed in Table 8.
Toble -8 Cascade filters
Multistage Median Filters: cascaded median filters
Median Hybrid Filters: cascaded altemating median and linear filters
Altema.tive Closing MIN(SHbnh(MAX(SHnbh »)
sequential Opening MAX(SHnbh(MIN(SHnbh »))
morphological
Close- MAX(SHnbh(MIN(SHnbh(MIN(SHnbh(MAX(SHnbh ))))))
filters opening
Open- MIN(SHnbh(MAX(SHnbh(MAX(SHnbh(MIN(SHnbh )))))))
c10sing
Quasi- o'k
= QSPREAD(Wnbh) =
spread filter R_ROS(Wnbh(SMTH(NBH )))- L_ROs(Wnbh(SMTH(NBH )))
"Wilcoxon
test"-fiIter o'k = MEAN(Wnbh(RANK(Wnbh )))
"Tamura's
test"-fiIter o'k = MEAN(Wnbh((RANK(Wnbh)Y))
"Median test
"filter ak ::: MEAN(Wnbh(sign(RANK(wnbh)- SIZE(Wnbh)/ 2)))

Computational expenses associated with iterative and cascade filtering in

conventional sequential computers may be reduced by using, as window
sampies in the process of scanning signal by the filtering window, those that
are already estimated in previous positions of the window. Two examples of
recursive filters are shown in Table 9. Recursive filters are not relevant for
parallel implementation.
524 Chapter 12

T.able -9. T wo examples 0 f recurslve NBH fil ters

Recursive
Median filters: "k = MEDN(RecWnbh)
Recursive
Algorithm for ak =a k .RECT(d)+[MEAN(RecWnbh)+ö 2 sign(d)].RECT(-d)

filtering impulse where d = Öl -Iak - MEAN(RecWnbh~ and Ö1 and Ö2 are

noise detection and correction thresholds,

12.2.4 Miscellaneous new filters

The presented classification eases analysis of structure of nonlinear filters

by reducing it to the analysis of what type of neighborhood, neighborhood
building and estimation operations they use. This analysis mayaiso lead to
new filters that "fill in" logical niches in the classification. Several examples
of such filters are given In Table 12-10. Examples of filter applications will
be illustrated in Sect. 12.3.
SizeEV-controlled Sigma-fllter improves image noise cleaning capability
of the "sigma"-filter (Table 12.5). Original "Sigma" filter tends to leave
untouched isolated noisy pixels whose EV-neighborhood is very small in
size if they substantially deviate from their true values. In SizeEV-
controlled Sigma-filter, size of EV-neighborhood is evaluated as an
additional attribute of EV-neighborhood and, if it is lower then a certain
threshold Thr, median over the window (or, in general any other of SMTH
operations) instead of MEAN (E V) is used to estimate window central pixel.
Size(Evnbh) is a useful attribute of EV-neighborhood that, by itself, can
be used to characterize image gray levellocal inhomogeneity, for example,
as "an edge detector", as it will be illustrated in Sect. 12.3
"Cardnl-filter" that generates image of local cardinality of its pixels can
be regarded as a special ca se of Size(Evnbh) filter for 8; = 8; = 0 . It can be
used for enhancement of small gray level inhomogeneity of images that are
composed of almost uniform patches.
P-histogram equalization generalizes local histogram equalization which
is its special case for P= 1. While histogram equalization enhances local
contrasts on a gray level q proportionally to histogram h(q), contrast
amplification coefficient for P-histogram equalization is proportional to
[h(q)Y , P-th power of h(q). When P=O, P-histogram equalization results in
automatic local gray level normalization by 10cal minimum and maximum.
For 256 quantization levels, it transforms image gray levels according to the
equation:
 12. NONLINEAR FILTERS IN IMAGE PROCESSING 525

a = 255 ak - MIN(NBH) (12.2.3)

k MAX(NBH)-MIN(NBH)

Table -10 Some new filters

Size controlled
a. =sw . MEAN(EVnbh(a. ))+ (1- sw )MEDN(SHnbh)
(SC-) "Sigma"- sw =RECT(SIZE(EVnbh(Wnbh;a. ;E;;E~ ))- Thr)
filter
Size_EV -filter ak = SIZE(EVnbh(Wnbh;E Vpl;E Vmmn ))
p- histogram equalization
a = f(h(q)Y '!(h(q)Y
k
q=O q=O

"Cardnl-filter" ak = HIST(Wnbh,ak)
NBH1-histo2ram equalization
EVnbh-histogram ak =RANK(EVnbh(Wnbh;ak;E Vpl;E Vmn))
equalization
KNVnbh-histogram ak = RANK(KNV(Wnbh;ak;K»
equalization
"SHnbh-histogram ak =RANK(SHnbh)
equalization

Spatially eonneeted (SC-) EV- and R-neighborhood filters

SC-K- ak = MEAN(CONCTD(KNV(Wnbh;ak;K~ak))
Nearest
Neighbor
filter
SC-"Sigma"- ak = MEAN(CONCTD(EVnbh(Wnbh;ak;EVpl;EVmn~ak))
filter
SC-Modified .. =
ak
Trimmed
Mean filters 1
MEAN(CONCTo(EVnbh(wnbh; MEDN(Wnbh }eVpl;e Vmn MEDN(Wnbh )))
SC-Alpha-
trimmed
at = MEAN(CONCTD(Qnbh(Wnbh, Rieft' R,ighl ~ MEDN(Wnbh )))
mean, ;
median
ak = MEDN(CONCTD(Qnbh(Wnbh,R"'ft,Rrighl~MEDN(Wnbh)))
Intermediate values of P enable flexible, with only one parameter P, eontrol
of image loeal contrast enhaneement. One of the immediate applieations of
P-histogram equalization is image dynamie range blind ealibration.
EV- , KNV- and SH-neighborhood equalizations represent NBH2-
histogram equalization. It is yet another generalization of the loeal
histogram equalization algorithm when it is earried out over neighborhoods
other then initial window neighborhood. Conventional loeal histogram
equalization and EV-neighborhood equalization are eompared in Fig. 12-11.
526 Chapter 12

v- and R- neighborhood based filters listed in Table 5 (K-Nearest

Neighbor, Sigma, Trimmed mean filters and alike) seleet from sampies of
the filter window those that are ''useful'' for subsequent estimation operation
by evaluation oftheir "nearness" to the selected sampIe in terms oftheir gray
levels and ranks. As one can see from Fig. 12-3, d - f, it certainly may
happen that the resulting neighborhood will contain sampIes that are not
spatially connected to the window central pixel. One can refme such a
selection by involving an additional check-up for spatial connectivity of
neighborhood elements. This is of especial importance in image denoising
applications.
 12. NONLINEAR FILTERS IN IMAGE PROCESSING 527

12.3 PRACTICAL EXAMPLES

In this section we provide illustrations of application and comparison of

nonlinear filters for image denoising, enhancement and segmentation.

12.3.1 Image denoising

In the design of filters for image denoising one should distinguish cases
of additive noise, impulse noise and of mixture of both types of noise. The
following equation describes the principle of nonlinear filtering for cleaning
additive noise:

alt)
kJ
= SMTH(NBH{a(t-l»)~
~ kJ ~, (12.3.1)

where SMTHO is one of the data smoothing estimation operations listed in

Tab. 12.2.
Because additive noise distorts signal gray values, iterative smoothing
over EV-neighborhoods is most natural. The simplest EV-smoothing is
"sigma"-filter (Table 12-5):

(12.3.2)

10 1\ I

8 \
6 \ --
4
- f\
2 I \ l
\1
2 3 4
Iteration

b)
a)

Figure 12-4. Iterative EV-neighborhood filtering additive noise: a) - test piece-wise constant
image (I), image distorted by additive noise with a limited dynamic range (2), filtered image
after 4 iterations (3), difference between noisy (2) and filtered (3) images amplified for better
visibility; b) - standard deviation of difference image as a function of the iteration index
 528 Chapter 12

Similar noise smoothing capability has a modification of "Sigma"-filter with

MEDN as a smoothing operation:

(12.3.3)

Being applied iteratively, these filters converge to a piece-wise constant

image, which is their root (fixed point) signal. This property enables very
efficient sUPl'ressing additive noise with values within a limited dynamic
range [c; ,c: ]
as it is illustrated in Fig. 12-4.

d) d)

Figure 12-5. Filtering additive noise with nonnal distribution a) - initial test piece-wise image
distorted by additive Gaussian noise with standard deviation 20 within image dynamic range
0-255; b) - result of 5 iterations of "Sigma" filter, with Wnbh=5x5 , c; c:
= = 20; c) -
result of 5 iterations of size-controlled "Sigma" filter (Eq. 12.3.4) with the same parameters
=
and threshold size Thr 5; d) filtering same image with median filter in the window of 5x5
pixels.
 12. NONLINEAR FILTERS IN IMAGE PROCESSING 529

Range [e; ,e; ] defines resolving power of EV-neighborhood smoothing

filters in image gray values. When additive noise has a distribution with
substantial tails outside this range, "Sigma"-filter and its median
modification tends to leave isolated noise outbursts outside the filter range.
For such outbursts, EV-neighborhood is very small and no averaging is
applied to them. This phenomenon is illustrated in Fig. 12-5, a) and b). As
one can see in Fig. 12-5, c), size-controlled "sigma"-filter described in Sect.
12.2.4 (Table 12-10)

a = SW • MEAN(EVnbh(a ))+ (1- sw )MEDN(SHnbh );

k k

sw = RECT(SIZE(EVnbh(wnbh; a k ; e;; e; ))- Thr) (12.3.4)

efficiently fixes this drawback of the "Sigma-filter". For comparison, Fig.

12-5, d) shows result of filtering same noisy image with a popular median
filter (Table 12-4):

a = MEDN(Wnbh)
k (12.3 .5)

As one can see, "Sigma-" and SC-"Sigma"-fiIters outperform median filter

very substantially in terms of both noise suppressing capability and
preserving image details.
The capability of SC-"sigma" filter to detect isolated noise outbursts may
also be effectively utilized for filtering mixture of additive and impulse noise
(see Fig. 12-6)

Figure 12-6. Iterative SC-"Sigma" filtering mixture of additive and impulse noise: a) a piece-
wise test image distorted by additive Gaussian noise with standard deviation 20 (in image
range 0-255) and by impulse noise with probability 0.15; b) SC "Sigma"-filtering result with
the same filter parameters as in Fig. 12-6.
530 Chapter 12

Figure 12-7. Filtering impulse noise: a) initial image; b) - image distorted by impulse noise
with probabilityO.3; c) -filtering result; d)

Detection of noise impulses is a key stage in filtering impulse

noise. For filtering impulse noise, one should first detect pixels
distorted by noise and then replace defective pixels by its estimate
obtained by smoothing over a pixel's neighborhood. This general
filtering principle is described by the equation

a(tl
k,l
= sw·a(t-I)
k,l
+ (l-sw) .SMTH es. (NBH';i-I))~
~ k,/ ~, (12.3.6)

where sw is a binary switch variable

sw = MEMB {SMTH dd (NBH'a(t-I)),\ a(/-I)}

~ k,l ~ k,l (12.3 .7)
 12. NONLINEAR FILTERS IN IMAGE PROCESSING 531

and SMTH ..t(NBH) and SMTHdet(NBH) are smoothing operation for

estimation and detection, respectively.
Performance of impulse noise filtering is characterized by the rate
of false detecting noise impulses (probability of false alarms), by the
rate of missing noise impulses (probability of missing) and by
measures of image distortions owing to false alarms, owing to noise
impulse missing and of distortions associated with replacement of
properly detected image sampies by their estimation. Fig. 12-7
illustrates these principles of impulse noise filtering on an example of
filter that uses MEAN operation over simple spatial neighborhood
Wnbh of 3x3 pixels:

SMTH ..I (NBH'a(t-I»)~=

~ k,l ~
SMTH dei (NBH'a(t-I»),\=
~ k,l ~
MEAN(Wnbh) •

(12.3.8)

12.3.2 Image enhancement.

Image enhancement is an image processing aimed at assisting visual

image analysis. Basically, two types operations are carried out for
image enhancement: amplification of local contrasts (Iocal contrast
enhancement land detection and enhancement of object boundaries,
which is conventionally referred to as edge detection.
Local contrast enhancement is amplification of a deviation of
imaged objects from background that eases visual object detection and
analysis. The deviation of objects from background is determined in
terms of different pixel attributes. One of the most natural attributes is
pixel gray value. Local contrast enhancement methods based on pixel
gray values belong to a family of methods that may be called "unsharp
masking" methods. These methods amplify difference between images
and their smoothed copies obtained by one or another image
smoothing operation:

(12.3.9)

When MEAN(Wnbh) is used as a smoothing operation, the method is

similar to aperture correction methods described in Sect. 8.2.3. Using
nonlinear smoothing operations reduces artifacts such as object
negative framing associated with mutual influence of objects and
image background onto the smoothing result that characteristic for
 532 Chapter 12

linear smoothing. Unsharp masking with linear and nonlinear

smoothing as a local contrast enhancement method is illustrated in
Fig. 12-8.

a) c)

Figure 12-8. Image unsharp masking using MEAN (Wnbh) and MEDN(Wnbh) in the
window of 15x 15 pixels as smoothing operations

Another useful attribute that may be used for emphasizing objects

with respect to their background is rank. Local contrast enhancement
methods based on pixel ranks are referred to as loeal histogram
equalization. The family of local histogram equalization algorithms is
described by the equation:

Qk = RANK(NBH,a k )· (12.3.10)

P-histogram equalization described in Sect. 12.2.4 adds flexibility to

these algorithms. Fig. 12-9 illustrates image local contrast
enhancement by local histogram equalization and P-histogram
equalization over spatial neighborhood Wnbh. P-histogram
equalization provides softer contrast enhancement and results in more
naturally looking images.
Local histogram equalization over other types of neighborhoods
mayaiso be useful. Fig. 12-10 illustrates local histogram equalization
carried out in windows of different spatial shapes.
Ordinary local histogram equalization over a spatial window and over
KNV- and EV-neighborhoods are illustrated in Fig. 12-11. One can observe
that latter two methods reveal some details that are invisible in the original
image and in the image obtained after local histogram equalization. It is
instructive to compare local histogram equalization with unsharp masking
shown in Fig. 12-8
 12. NONLINEAR FILTERS IN IMAGE PROCESSING 533

Figure 12-9. Loeal histogram equalization and P-histogram equalization: a) - initial image
(256x256 pixels) of eye blood vessels; b) - loeal histogram equalization over spatial window
of 35x35 pixels; e) - P-histogram equalization with P=O.5.
 534 Chapter 12

a) b) e) d)

Figure 12-10. Loeal histogram equalization over Shape-neighborhoods: a) - initial image of a

printed eireuit board; b) - d) - loeal histogram equalization over spatial windows of 35x35,
lx35 and 35xl pixels, respeetively.

e) d)

Figure 12-11. Comparison ofloeal histogram equalization over spatial neighborhood and over
EV - and KNV -neighborhoods applied to an angiogram. A) - initial image (256 quantization
levels, 256x256 pixels); b) loeal histogram equalization over spatial window of 15xl5 pixels;
e) - loeal histogram equalization over KNV-neighborhood in the same spatial window with
K= 113 (half of the window size); d) loeal histogram equalization over EV -neighborhood in
=
the same spatial window with 8; 8; 10. =
 12. NONLINEAR FILTERS IN IMAGE PROCESSING 535

Other pixel attributes that may be useful for visual image analysis are size
ofEV-neighborhood and pixels cardinality. Image enhancement by means of
measuring and displaying these attributes is illustrated in Fig. 12-12 on an
example of a microscopic image of crystals.

Figure J2-12. Local contrast enhancement using Size-EV and cardinality as pixel attributes:
a) - original image (256 gray levels); b) -c): Size-EV (C; =C; =3) and pixel cardinality
displayed as images.

12.3.3 Image segmentation and edge detection

Image segmentation and edge detection are interconnected types of

image processing. Image segmentation is aimed on segmenting image into
non-overlapping areas according to their certain features. Provided
appropriately selected features, segmentation separates different objects in
images. Segmentation results in piece-wise constant images in which pixel
values are indices of segmented areas. In this sense it is an operation akin to
image smoothing for piece-wise constant model of images. This smoothing
eliminates small variations of object attributes, selected as segmentation
features. As a result of smoothing for segmentation, image histogram
degenerates. Its modes that correspond to different segments are becoming
sharper and narrower in the process of iterative smoothing and tend to
converge to a histogram that consist of a set of delta-functions. Fig. 12-13
iIlustrates image segmentation by iterative smoothing and the phenomenon
of degenerating image histograms. Smoothing was carried out by the
algorithm:
 536 Chapter 12

(12.3.11)

in the window of 5x5 pixels with c; = c: = 3 (image for 256 quantization

levels).

15 - - I-
_L __ L __ ""' __ -I _ _
I I I I I
I I
1 - _!..
I
I

05 - - I"
I

c)

e) f)

Figure 12-13. Image segmentation by means of iterative smoothing: a) - original image; b) -

smoothed image after lO iterations of the algorithm of Eq. 12.3.11; c) - gray level histogram
of the original image; d) - gray level histogram of the smoothed image; e) and f) - image
segments that correspond to two extreme modes of histogram d)

Edge detection is outlining object boundaries. In principle, edge

detection should be preceded by image segmentation. In fact, edge detection
in perfect1y segmented image is a trivial task of marking boundaries between
 12. NONLINEAR FILTERS IN IMAGE PROCESSING 537

image segments. However in practice of image processing edge detection is

frequently considered as a primary task and is carried out by means of
measuring deviations in one or another pixel attribute from those of
neighboring pixels. In this respect edge detection methods are akin to the
above described methods of local contrast enhancement. F ig. 12-14
iIIustrates edge detection algorithms that use three types of neighborhood
attributes: standard deviation within spatial window STDEV(Wnbh),
SEIZE(EV(Wnbh)) and MAX(Wnbh)-MIN(NBH).

a) b)

c) d)

Figure 12-14. Edge detection using different image attributes: a) -Iocal standard deviation in
the window of 7x7 pixels; b) - local standard deviation in the window of 3x3 pixels; c) -
SIZE(EV(Wnbh) in the window of 7x7 pixels, 8; = 8; = 3 (256 level of quantization); d)
- differenee between loeal maxima and minima over spatial window of 3x3 pixels. Image c) is
identieal to the image ofFig. 12-12 b) shown in negative.

Fig. 12-15 iIIustrates the importance of image smoothing before edge

detection. Compare magnified image fragments to see that smoothing image
before applying edge detection processing enables preserving fine image
details that otherwise are destroyed by noise in the original image.
538 Chapter 12

Figure 12-15. The importance ofimage smoothing before edge detection: a) initial image ofa
printed circuit board; b) - edge image generated by the algorithm MAX(Wnbh3x3)-
MIN(Wnbh3x3; c) - edge image generated by the same algorithm after image smoothing by
Sigma-filter (MEAN(EV(Wnbh5x5,3,3))); d), e) - corresponding magnified fragments of the
edge images.

12.3.4 Implementation issues

Described nonlinear filters are inherently parallel. Therefore, the most

natural way to implement them is to use multi layer parallel networks with
neuro-morphic structure. Each level of such a network contains an array of
elementary processors that implement estimation operations described in
Sect. 12.1.3. The processors in each layer have to be are connected with
corresponding sub-arrays of processors on the previous layer that form their
neighborhood according to the principles described in Sect. 12.1.4. For N
stage filters, there should be N such layers. The last layer in the structure
produces filter output. In each layer, processors work in parallel and process
"neighborhood" pixels formed on the previous layer to produce output for
the next level or finally, the filter output. One can not avoid mentioning that
such a multi layer structure much resembles the structure of visual retina in
animal and human eyes.
12. NONLINEAR FILTERS IN IMAGE PROCESSING 539

Modem advances in "smart pixel arrays" promise a possible electronic

implementation of such an artificial retina. Another option is associated with
opto-electronic implementations that are based on the natural parallelism of
optical processors ([14]).
In serial computers, software implementation of the filters may be based
of efficient recursive algorithms for computing local histograms and order
statistics described in Sect. 7.1.1.
540 Chapter 12

References
1. J. W. Tukey, Exploratory Data Analysis, Addison Wesley, 1971
2. J. Serra, Image Analysis and Mathematical Morphology, Academic Press, 1983, 1988
3. V. Kim, L. Yaroslavsky, "Rank algorithms for picture processing", Computer Vision,
Graphics and Image Processing, v. 35, 1986, p. 234-258
4. I. Pitas, A. N. Venetsanopoulos, Nonlinear Digital Filters. Principles and Applications.
Kluwer, 1990
5. E. R. Dougherty, An Introduction to Morphological Image Processing, SPIE Press, 1992
6. H. Heijmans, Morphological Operators, Academic Press, 1994
7. E. R. Dougherty, J. Astola, An Introduction to Nonlinear Image Processing, SPIE
Optical Engineering Press, 1994
8. L. Yaroslavsky, M. Eden, Fundamentals 0/Digital Optics, Birkhauser, Boston, 1996
9. J. Astola, P. Kuosmanen, Fundamentals o/Nonlinear Digital Filtering, CRC Press, Roca
Baton, New York, 1997
10. E. Dougherty, J. Astola, Nonlinear Filters/or Image Processing, Eds., IEEE publ., 1999
1l. L. P. Yaroslavsky, K. o. Egiazarian, J. T. Astola, "Transform domain image restoration
methods: review, comparison and interpretation", Photonics West, Con/erence 4304,
Nonlinear Processing and Pattern Analysis, 22-23 January, 2001, Proceedings 0/ SPIE,
v.4304.
12. D. L. Donoho, I.. M. Johnstone, "Ideal Spatial Adaptation by Wavelet Shrinkage",
Biometrica, 81(3), pp. 425-455,1994
13. P. D. Wendt, E. J. Coyle, and N. C. Gallagher, Jr., "Stack Filters ", IEEE Trans. On
Acoust., Speech and Signal Processing, vol. ASSP-34, pp. 898-911, Aug. 1986.
14. T. Szoplik, Selected Papers On Morphological Image Processing: Principles and
Optoelectronic Implementations, SPIE, Bellingham, Wa., 1996 (MS 127).
 Chapter 13
COMPUTER GENERATED HOLOGRAMS

13.1 MATHEMATICAL MODELS

Basic stages in the synthesis of computer generated holograms are: (i)
formulating mathematical models of the object and of the usage of the
hologram; (ii) computing the mathematical hologram, or distribution of the
wave front amplitude and phase in the hologram plane; (iii) encoding
sampies of the mathematical hologram for recording them on the physical
medium; (iiii) fabrication ofthe hologram (Fig. 13-1).
Mathematical model of the object is intended to specify the amplitude
and phase distribution of the object wave front. Two types of the models can
be distinguished: 2-D or 3-D arrays of points (object sampies) and models
that represent objects as compositions of elementary diffracting elements
such as point scatters, segments of 2-D or 3-D curves, slits, etc. The former
is more general. It requires specifying amplitudes and phases of the object
wave front in all points ofthe array.
Mathematical models of usage of to be generated holograms describe
geometry of wave propagation from objects to the hologram plane and
specify criteria for performance evaluation of the hologram.
Computing the mathematical hologram simulates numerically wave
propagation for the selected models of the object and of the hologram usage.
Mathematical holograms are arrays of complex numbers that represent
amplitudes and phases of hologram sampies. For fabrication of holograms
these numbers should be converted into an array of numbers that control
optical properties of the physical recording medium used for recording the
hologram. We will refer to this conversion as to hologram encoding.
 542 Chap/er 13

r - - - - - - - - , ~ :: ';': ~ . ~:-:-: ::.:. :~ ::: ~: ~ .: -:':!:,. ~:~ .; . ~:~ :1

:~ Complex ... :t
MatbematicaJ ~. amplitude of .:....i. :~
modeloftbe
object
T objed wave ~ Computing the ~. ~
I :. field mathematical'
L...._ _ _ _... i:. .:: hologram :;

Hologram rT~ :~ ~ ~ ~ T : ::HU: : : : : ~~

osage model :) T~ k~~~~~ : )~j t· ....... ..,., .. ..:,
1..:, . ,;..· ...·.,,;.. :,.·..,; ..;.. .... ·..,; .-' ....... -
::
ri :'-: : - :_.- --- _.- - _.- - _.- --- - · - :~f : - -_._-
: : : : : : .........
: : : : :............
:: : : .. .. ..... ....... '. ' .... ... ..... : ~
I:.:.:.:.:.:.:.:.:.: Encoding computer generated ltoJogram :I
! P!g!t~l~~
. :. :. :. for rec:ordinK .,
:::::~~~~~~: ::::::.:.:.:.:.:.:.:.:.:.:.:.:.:1 L· : .: . :.: . :. : . : . : . : . :.:.
t .: . : . : . : . : . : Recording
. : . : . : .computer generated hologram . i
':-:-:-:-:-:-: -: -:-:
... _ . ...... ~. ~ ... 0 __ ....... . ~ ...
.,
~. ..... "_" ... ... " ..J ~ . . . . . . . . . . "'- " .... i6. ' ~ ....

Figure 13-1. Basic stages in synthesis of computer generated holograms

Major applications of computer generated holograms are spatial filters for

optical infonnation processing, computer generated optical elements such as
beam fonners , deflectors and special diffraction gratings, and infonnation
display. Mathematical models for computer generated spatial filters such as,
for instance, filters for implementing optimal adaptive correlators discussed
in Chapt 11 are defined by the filter frequency response. Mathematical
models for computer generated optical elements are directly fonnulated by
the requirements to the specific optical elements. In this chapter we will
describe a mathematical model for computer generated display holograms.
Consider a schematic diagram for the visual observation of objects
shown in Fig.13-2. The observer's position with respect to the observed
object is defined by the observation surface where the observer's eyes are
situated. The set of observation foreshortenings is defined by the object
observation angle. In order that the ob server may see the object at the given
observation angle, it suffices to reproduce, by means of a hologram, the
distribution of intensities and phases of the light waves scattered by the
object on the observation surface.
Consider monochromatic object illumination, wh ich enables one to
describe light wave transfonnations in tenns of the wave amplitude and
phase. Object characteristics defining its ability to reflect and scatter incident
radiation may be described for our purposes by a radiation reflection
factor with respect to the light intensity O(x,y,z) or amplitude o(x,y,z)
 13. COMPUTER GENERA TED HOLOGRAMS 543

which are functions of the object's surface coordinates (x, y, z) and are
connected with the relationship: O(x,y,z)= lo(x,y,zf .

C 1. ~ __
_---,- ----------
..
lIIum'",,"on

---- --------- ;7 \
/-~7_--.
\
. ,-- ---- ---- \
,,,
\

- - --- --- --- - ......"

\
\
\
I
I
I

--.',
Light
scattered by
angle the object surface

Figure 13-2. Scheme ofobservation ofholograms

The intensity of the scattered wave 1 (x, y, z) and its complex 0

amplitude Ao(x,y,z) at the point (x,y,z) are related to the intensity

I(x,y,z) and complex amplitude A(x,y,z) of the incident wave as
follows:

1 0 (x, y,z)= O(x,y, z)I(x,y, z)

(13.1.1)
Ao(x,y,z)= o(x,y, z)A(x,y, z) .

The amplitude reflection factor o(x, y, z) may be regarded as a

complex function of the spatial coordinates represented as

o(x, y, z)= lo(x, y, z~ exp[ie (x, y, z)].

0 (13.1.2)

Its modulus lo(x,y,z~and phase eo(x,y,z) show how the modulus

IA(x,y,z)1 and the phase ro(x,y,z) of
the complex amplitude
IA(x,y,z)lexp[iro(x,y,z)] ofincident light are changed after reflection by
the object surface at the point (x, y, z):
544 Chapter 13

lAD (X,y,z~ = IA(X,y,Z~IO(X,y,Z~

(13.1.3)
CO (X,y,Z)=CO(X,y,z)+9" (X,y,Z),
0

where IA,,(x,y,Z~ andco,,{x,y,z) are, respectively, the modulus and the

phase of the reflected wave front.
The relation between the complex amplitude r(;, 11 ,l;) of the light wave
field over an arbitrary surface defined at the coordinates (; ,11 ,l; ) and the
complex amplitude A" (x, y, z) of the wave front at the object surface can by
described by a wave propagation integral over the object surface S"

r(~ ,11,l;)= JJJA" (x,y,z)T(x,y, z;~,11 ,l; }dxdydz , (13.1.4)

s.

whose kernel T(x, y, z;;, 11 ,l;) depends on the spatial disposition of the
object and the observation surface. Reconstruction of the object can be
described by a back propagation integral over the observation surface S s :

Ao(x,y,Z)= JJJr(~,11,l; )T(~,11,l; ;x,y,z)i~d11dl; , (13.1.5)

s,

where Ao (x,y,z) is complex amplitude of the object as it is seen from the

observation surface and f(~, 11 ,l; ; x, y, z) is a kerne I reciprocal to
T(x, y, z;~,11 ,l; ) .
Thus, the hologram synthesis lies in the computation of r(~,,, ,l; ) through
A" (x, y, z) which is to be defined by the object description and illumination
conditions, and in recording the results on a physical medium in a form
allowing interaction with radiation for visualizing or reconstructing
A,,(x,y,z) according to Eq. (13.1.5).
3-D integration required by Eq. 13.1.4 can be reduced to much more
simple 2-D integration by taking into consideration the following natural
limitations ofvisual observation:
• The pupil of the observer's eye is usually much smaller than the
distance to the observation surface.
• The area of the observation surface is large as compared to the inter-
pupil distance and the surface may be regarded as flat.
• The depth of objects situated at a distance convenient for observation
usually are small as compared with the distance.
 13. COMPUTER GENERA TED HOLOGRAMS 545

In view of these limitations, one may break down the observation surface
into areas approximated by planes, and, using the laws of geometrical
optics, replace the wave amplitude and phase distributions over the object
surface by those of the wave in a plane tangent to the object ( or sufficiently
elose to it, so as to make diffraction effects negligible) and parallel to the
given plane area of the observation surface. In this way the problem of
hologram synthesis over the whole observation surface boils down to the
synthesis of fragmentary holograms for the plane areas of this surface, with
the complete hologram being composed as a mosaic of fragmentary ones.
Instead ofEq.(13.1.4) one then obtains for fragmentary holograms:

f(~, Tl) = JJA (x, y)TD(x, y;~, Tl }dxdy ,

o (13.1.6)
s.

where Ao(x, y) = IAo (x, y ~ expliro (x, y)

0 is a complex function resulting
from recalculation of the amplitude and phase of the field reflected by the
object onto the plane (x, y) tangent to it and parallel to the observation plane
(~,Tl);TD(X,y;~,Tl) is a reduced 2-D transformation kernel for the distance
between these planes D .
If the geometrical dimensions of the object under observation and of the
observation surface fragment are small as compared with the distance D
from the object to the observation plane, the integral relationship of Eq.
13.1. 7 is, as it was shown in Sect. 2.3.2, reduced to integral Fresnel
transform:

where A. is the illumination wave length. Discrete representation of the

integral Fresnel transform suitable for its numerical computing is discussed
in Sect. 4.4. The holograms synthesized by means ofthis relationship will be
referred to as synthesized Fresnel holograms.
Further simplification is possible if

(13.1.8)

in which case the integral ofEq.(13.1.8) can be approximated as

-00-00

(13.1.9)
546 Chapter 13

through the integral Fourier transform:

JJA (x, y)exp[i21t (x~ + J'11 )/W]dxdy .

<Xl <Xl

rA~, 11) = o (13.1.10)

-00-00

The holograrns synthesized according to Eq. 13.1.10 by means ofFourier

transformation will be referred to as synthesized Fourier holograms. Fourier
and Fresnel holograms can be computed through Discrete Fourier transforms
discussed in Sect. 4.3 and 4.4.
Schemes of using computer generated display Fresnel and Fourier
holograms for visual observation ofvirtual objects are shown in Fig. 13-3, a)
and b), respectively.

Point source of
Reconstructing reconstructing spherical
parallel light beam lightbeam

- -----------~
-------------~
--------------~

Hologram Hologram

Figure J3-3. Scheme of visual observation of virtual objects using computer generated
Fresnel (a) and Fourier (b) holograms

From the standpoint of object wave front reconstruction, Fresnel

holograms differ from Fourier holograms in that they have focusing
properties and are capable of reproducing the finite distance from the
observation surface to the object. For reconstruction of Fourier holograms,
spherical reconstruction light beam is needed and reconstructed image is
observed in the plane of the reconstructing beam point source.
Mathematically, reconstructing an object by Fresnel and Fourier
holograms is described by the inverse Fresnel and Fourier transformations,
respectively. When holograms are observed visually, these transformations
are performed by the optical system of the eye.
 13. COMPUTER GENERA TED HOLOGRAMS 547

Having passed from the spatial 3D-problem to the problem of treating its
20 approximation, we, strictly speaking, lose the possibility of taking into
account the precise effect of object depth relief on the wave front. Even
Fresnel holograms use only a single distance value from the object to the
observation plane rather than object relief depth. Nevertheless, it is still
possible to synthesize holograms which are capable of reconstructing 3-D
images for visual observation and retain the most important property of
holographic visualization - naturalness of object visual observation. At
least two options exist for this: composite computer generated stereo- and
macro-holograms and programmed diffuser holograms,
Stereo-holograms are pairs of holograms synthesized to be viewed by left
and right eye and to reconstruct the corresponding stereo views of the object
([I]). Composite stereo holograms are composed of multiple holograms that
reproduce different aspects of the object as observed from different
positions. Macro-holograms are large composite stereo-holograms that
cover wide observation angle and can be used as a wide observation window.
An example of a composite stereo-hologram that cover full 3600 of
observation angle with horizontal parallax is shown in Fig. 13-4.
Programmed diffuser holograms are computer generated Fourier
holograms that imitate properties of diffuse surfaces to scatter irradiation
non-uniformly in the space and in this way provide visual clue about the
object surface shape (See Sect. 7.2.4).

Figure 13-4. Composite stereo maero hologram, a holographie "movie", eomposed of 800
e1ementary holograms (adopted from [I)). I - holograms reeorded on a photographie film, 2 -
annular holder, 3 - eonie refleetor. Holograms are iIIuminated from above through the eonie
refleetor and are viewed as through a window.
548 Chapter 13

13.2 METHODS FOR ENCODING AND RECORDING

COMPUTER GENERATED HOLOGRAMS

Optical media for recording computer generated holograms may be

classified into three categories: amplitude-only media, phase-only media,
and combined amplitude/phase media.
In amplitude-only media, the controlled optical parameter is the light
intensity transmission or reflection factor. This is the most common and
available class, whose typical representatives are the standard silver galid
photographic emulsions used in photography and optical holography ([2]).
In phase-only media, the optical thickness of the medium can be
controllable, for example, by varying the medium refractive index, or
physical thickness, or both. Phase-only media include thermoplastic
materials, photo resists, bleached photographic materials, media based on
photo polymers, etc. Recently, micro-Iens array and micro-mirror array
technology has emerged ([3,4]) that offer new opportunities for recording
computer generated holograms.
Combined media allow independent control of both the light intensity
transmission factor and of the optical thickness. Currently, these are
photographic materials with two or more layers sensitive to radiation of
different wavelengths. This permits the user to control of the transparency of
certain layers and the optical thickness of others by exposing each layer to
its wavelength independently.
Special digitally controlled hologram recording devices are required for
modulating optical parameters of these media according to the mathematical
hologram. No such special purpose devices exist as yet, and computer printer
and display devices designed for the output of characters, graphs, and gray-
scale and color images are used instead. The distinctive feature of many of
such devices is that they perform only binary or two level modulation of the
medium's optical parameters. Amplitude-only and phase-only media whose
controlled optical parameters may assume only two values will be referred
to as binary media. The use of amplitude and phase media in the binary
mode is quite inefficient in terms of the medium information capacity
because the possibility of writing information on them is defined only by
their spatial degrees of freedom (their resolution power), whereas in
amplitude and phase media, in principle, the degrees of freedom related to a
transmission (reflection, refraction) factor may be used as weIl. The major
advantage of the binary mode is simplicity of media exposure, photo-
chemical processing when it is needed and copying.
The most important characteristics of recorders are their sampling step;
that is, the distances A~ and AT) between neighboring, separately exposed,
resolution cells, and the total number of cells which may be exposed. The
 13. COMPUTER GENERA TED HOLOGRAMS 549

sampling step defines the angular dimensions ofthe reconstructed image. For
instance, in order to make the image's angular dimensions about 100 or
more with a reconstruction light wavelength of about 0.5 ~, A~ and AT)
should be 3 ~ at most. The existing recording devices have a sampling step
of about 1 through 10 ~, and the total number of resolution cells, by the
order of magnitude, from 10 3 x 10 3 to 10 4 X 10 4 • At the early stages of
digital holography, holograms were recorded by means of standard computer
plotters and printers. The obtained hologram hard copies were then
photographically reduced in order to achieve acceptable values of the
sampling step ([1, 5,6]).
The existing methods for recording synthesized holograms on amplitude-
only, phase-only, binary, and combined amplitude/phase media may be
classified with respect to different features. Methods of representing complex
numbers describing sampIes of the mathematical hologram will be chosen
here as the classification key (Fig. 13-5). For other possible classifications
the reader is referred to the reviews [5-7].
Complex numbers may be represented in two ways: in the exponential
form A expVcI» where A and cl> are, respectively, the amplitude and phase
of the numbers or additively as a sum of two or more basis complex
numbers. Obviously, for hologram recording on the combined
amplitude/phase media, the exponential form is the most suitable. It does not
require any special encoding. Computer generated holograms recorded on
combined media are capable of wave front reconstruction on optical axis of
the reconstruction set-up at the zero-order diffraction order and are called
on-axis holograms.
For recording on phase-only or amplitude-only media, special hologram
encoding methods are required. One of the most straightforward encoding
methods oriented on using phase-only media is that of kin%rm ([8,9]).
Kinoform is a computer generated hologram in which amplitude data of the
mathematical hologram intended for recording are disregarded and only the
hologram sampIe phases are recorded on a phase-only medium. Although
disregarding hologram sampIe amplitudes results in substantial distortions
such as appearance of speckle noise in the reconstructed wave front (see
Sect. 7.3), kinoforms are advantageous in terms of the energy of
reconstruction light because the total energy of the reconstruction light is
transformed into the energy of the reconstructed wave field without being
absorbed in the hologram. Moreover, the distortions may to some degree be
reduced by an appropriate choice of the diffusion component of the wave
field phase on the object using an iterative optimization procedure similar to
that described in Sect. 7.2.4 ([6]).
 550 Chapter 13

E.lponentia) Combined phase/ "On axis"

'---- holograms
repretentadoll of amplitude media
eomple.l ampUtude of

I
tbe wave field

I
Phase-only
media I I " Kinororm"

Encoding by
Amplitude- I .,' "symmetrization"
, ' ,
onlymedia ~
Binary media 1' ", "Detour phase"
.- I ' method
,* " , ,

Addjtive Double-phase
representation Phase-only method
orcomplex media
amplitude of Multiple phase
the wave Held method
Binary
media Representation by
orthogonal and bi-
orthogonal
components

Ampljtude-
only
media
~ - 2-D symplex
representation

~ Explicit spatial
carrier methods

Figure 13-5, Classification of methods for recording computer generated holograms

In hologram recording on amplitude media, the main problem is recording

the phase component of hologram sampies. The symmetrization method
([I]) offers a straightforward solution of this problem for recording Fourier
holograms. The method assumes that prior to hologram synthesis, the object
be symmetrized, so that, according to properties of Fourier transform, its
Fourier hologram contains only real valued sampies and, thus, may be
recorded on amplitude-only media. As real numbers may be both positive
and negative, holograms should be recorded in the amplitude-only medium
with a constant positive bias, making all the recorded values positive.
For SDFT as a discrete representation of the integral Fourier transform,
symmetry properties (see Sect. 4.3 .2) require symmetrization through a rule
depending on the shift parameters. For example, with integer 2u and 2v the
 13. COMPUTER GENERA TED HOLOGRAMS 551

symmetrization rule for the object wave front specified by the array of its
sampIes {Au}' k = O,I,•••,N) -I, 1 = O,I,•••,N1 -I becomes

(13.2.1)

which implies symmetrization by object duplication. In doing so, the number

of sampies of the object and, correspondingly, its Fourier hologram, is twice
that of the original object. It is this two-fold redundancy that enables one to
avoid recording the phase component. Symmetrization by quadruplicating is
possible as weIl. This consists in symmetrizing the object according to the
rule of Eq.(13.2.1) with respect to both indices k and I. Hologram
redundancy becomes in this case four-fold. Holograms of symmetrized
objects are also symmetrical and reconstruct duplicated or quadruplicated
objects depending on the particular symmetrization.
Notably, duplication and quadruplication do not imply a corresponding
increase in computer time for execution of SDFT at the hologram calculation
step because, for computation of SDFT, one may use combined algorithms
making use of signal redundancy for accelerating computations (see
Sect.5.3).
Historically, one of the first methods for recording synthesized holograms
was that proposed for amplitude-only binary media by A.Lohmann and his
collaborators ([ 10, 11]). In this method, an elementary cell of the medium is
allocated for reproducing the amplitude and phase of each sampie of a
mathematical hologram. The modulus of the complex number is represented
by the size of the opening (aperture) in the cell and the phase - by the
position of the opening within the cello
All the cells corresponding to mathematical hologram sampies are
arranged over a regular (usually, rectangular) raster. A shift of the aperture
by Af in a given cell with respect to its raster node corresponds to a phase
detour for this cell equal to (21tAfcos9 }/A. for hologram reconstruction at
an angle 9 to the system's optical axis perpendicular to the hologram plane
(Fig.13-6), where 1.. is wavelength of the illumination light used for
hologram reconstruction. The use of the aperture spatial shift for
representing complex number phase was named the detour phase method
([6])
 552 Chapter 13

Encoded computer
~enerated holo~ram

Nodes ofthe
\ '. discretization raster
\ '.
\
"'\ \ \ '.'.\
\
\ \
\ \
,
\ \

\ \
~\. \,

"...
\ \

~ ~ ~ ~
Direction to the Direction of image
observation plane reconstruction

Figure 13-6. Detour phase method for coding phase shift by spatial shift ofthe transparent
aperture

As was already noted, only spatial degrees of freedom are used for
recording in binary media; therefore, the number of binary medium cells
should exceed the number of hologram sampies by a factor equal to the
product of the amplitude and phase quantization levels. This product may run
into several tens or even hundreds. The low effectiveness of using the
degrees of freedom of the hologram carrier is the major drawback of the
binary hologram method. However many modern technologies such as
lithography and laser and electron beam pattern generators ([ 12]) provide
much higher number of spatial degrees of freedom that the required number
of hologram sampies. Therefore this drawback is usually disregarded and
such merits ofbinary recording as simpler recording and copying technology
of synthesized holograms and the possibility ofusing commercially available
devices made it the most widespread. Numerous modifications of the method
are known, oriented to different types of recording devices.
With an additive representation, the complex number (regarded as a
vector in the complex plane) is a sum of several components. For recording
on amplitude-only media these components should be assigned standard
 13. COMPUTER GENERA TED HOLOGRAMS 553

directions on a complex plane (phase angle) and controllable length

(amplitude ).

+,
Vif
- ~ r - ~ r eo
e;m~ e~~I

____ I
I
_L.
_
I I
. . . __ 1 _J_.
I
I
I
I

k'-:e
- I _ /" , I

•
I e re e-re I e re /" 'I

I I _'~ ----
I 240 e120
I

a) b) e) d)

Figure 13-7. Additive representation of eornplex nurnbers over orthogonal basis (a), over
biorthogonal basis (b), over 2-D syrnplex (e) and as a surn oftwo equallength veetors (d).

The simplest case is the representation of a vector r by its orthogonal

components, say, real r,e and imaginary r;m parts:

(13.2.2)
where e re and e im are orthogonal unit vectors. When recording a hologram,
the phase angle between the orthogonal components may be encoded by the
detour phase method, rre and r;m being recorded into neighboring
hologram resolution cells neighboring in araster rows as shown in Fig.13-7,
a. The reconstructed image will be then observed at an angle e~ to the axis
defined as folIows:
(13.2.3)

For recording negative values of rre and r;m a constant positive bias may
be added to recorded values. We will refer to this method as to orthogonal
encoding method.
Since two resolution elements are used here for recording one hologram
sampIe, such a hologram has double redundancy, as in the symmetrization
with duplication.
With such encoding and recording of a hologram, one must take into
account that the optical path differences A. / 2 and 3A. /4 will correspond to
the next pair of hologram resolution cells (see Fig.13-8 a); that is, values of
r,e and r;m for each odd sampIe of the mathematical hologram should be
recorded with opposite signs.
554 Chapter 13

s r r+l s r

ere elm -ere -elm ere elm e-re e-

Im
0 0 900 180 0 2700 0 0 900
180 0 2700

mo=O mo=1

a) b)
r r+l

ere e/m e-
re
e.,--
,m no=O
0 0 90 0 180 0 2700
s
e-n e.,--
,m ere elm
00 900
no=1
180 0 270 0

mo=O mo=l
c)
Figure 13-8. Hologram encoding by decomposition of complex numbers in orthogonal (a) and
biorthogonal (b, c) bases. Hologram encoding cells corresponding to one hologram sampIe is
outlined with a double line.

This encoding technique may be described formally as folIows. Let {r,s}

be indices of mathematical hologram sampIes {rr.s}' r = O,I"",M -1,
{m,n} be indices ofthe recording medium resolution cells, be sampIes {fr.• }
of the encoded hologram ready for recording and let index m is counted as a
two digit number:

m=2r+m o, m o =0,1, (13.2.4)

and index n is counted as n = s . The encoded hologram can then be written

as

(13.2.5)

where b is a positive bias constant needed to avoid recording negative

values.
In reconstruction of computer generated holograms recorded with a
constant bias, substantial part of energy of the reconstructing light beam is
not used for image reconstruction and goes to the zero-order diffraction spot.
 13. COMPUTER GENERA TED HOLOGRAMS 555

One can avoid the constant biasing for recording r,e and r im byallocating,
for recording one hologram sampie, four neighboring-in-raster medium
resolution cells rather then two ([5]). This arrangement is shown in Fig.13-8,
b. With a reconstruction angle as defined by Eq.(13.2.3), phase detours 0,
1t / 2, 1t and 3n: / 2 correspond to them. Therefore, cells in each group of
four cells allocated for recording of one sampie of the mathematical
hologram should be written in the following
1- 1-
order: (r,e + Ir,el)/2; (rim + Irim j)/2; ~r,e r,e )/2; ~rim r im )/2. One can
see that in this case all the recorded values are nonnegative. The method
represents a vector in the complex plane in abiorthogonal basis
(e,e ,e-Fe ,e im ,e:-):
Im

(13.2.6)

In terms of indices (m,n) of recording medium resolution cells counted

as

m = 4r + 2m ol + m 02 ; r = 0,1,..•, M - 1; mOl = 0,1; m 02 = 0,1 ; n = s ,

(13.2.7)

this encoding method may be described as follows:

r =!4 f;m02 [(_1)m o2 r +r' J+ (_I)mol imo21 (_1)mo2 r +r' I.

m,n ~ ~ ',S1,5 ',S ',S
(13.2.8)

In hologram encoding by this method, it is required, that the size of the

medium resolution cell in one direction be four times smaller than that in the
perpendicular one. In this way the proportions of the reconstructed pieture
can be preserved. A natural way to avoid this anisotropy is to allocate pairs
of resolution cells in two neighboring raster rows for each sampie of the
mathematical hologram as it is shown in Fig.l3-7, c); that is, to record the
hologram sampies according to the following relationship:

rm,n
=!4 {;mo r[(_l)mor +r' ]+ (_l)noi moo 1(_l)mor +r'
',S 1,S ',S
1 (13.2.9)
1,S'
556 Chapter 13

where indices (m, n) are counted as two digit numbers:

m = 2r + m o ; n = 2s + n o; m o = 0,1 ; n o = 0,1 . (13.2.10)

With this encoding method, images are reconstructed along a direction

making angles S~ and Sn with axes (1;,1')) in the hologram plane defined
by the equations:

L\1;cosS~ =').../2; L\1')cosS n =').../2. (13.2.11)

Representation of complex numbers in the biorthogonal basis of

Eq.(13.2.6) is redundant because two of the four components are a1ways
zero. This redundancy is reduced in the representation of complex numbers
with respect to a mo-dimensional simplex (e o , ello , e240 ) (Fig.13.6, c):

(13.2.12)

Similarly to the biorthogonal basis, this basis is also not linearly

independentbecause

and its redundancy is exploited to insure that components {fo,fprJ of

complex vectors be non-negative.
Two versions of hologram coding by the two-dimensional simplex are
known. In the first version, complex vectors in the complex plane are
represented as the sum of two components directed along those of three
vectors (e o ' ello , e240 ) which confine the third part of the plane where this
vector is situated ([13]). It follows that of the three numbers {fo,fl ,f2 }
defining vector r through Eq.(13,.2.2), two are always non-negative and are
projections of the vector r on the corresponding basis vectors, and the third
one is zero. The following relations may be obtained for components
{fo,fl ,f2 } from this condition

f o = [(1 + signAXB -IBI)+ (1- signAXC + ICI)V2~;

f l = [(1 + signCXA -IAI)+ (1- signCXB + IBI)V2~;

f o= [(1 +signBXC -ICI)+ (1- signBXA +IAI)V2~, (13.2.14)
 13. COMPUTER GENERA TED HOLOGRAMS 557

where

A = (r + r* )/2;

B = [r exp(i2n /3)+ r* exp(- i2n /3 }}!2 ;

C = [r exp(- i2n /3)+ r* exp(i2n /3 }}!2, (13.2.15)

where signX = X/lXi.

The second version of the method makes use of the fact that, by virtue of
Eq. 13.2.13, the addition of an arbitrary constant to {ro, rp r z } leaves Eq.
13.2.12 unchanged. It represents {ro,rt,rz }as

{r, = r,(O) + V}, t = 0,1,2 (13.2.16)

The presence of an arbitrary constant V allows to impose on {r}O)}; t = 0,1,2
an additional constraint:
r:O) + rt(O) + r~O) = 0 . (13.2.17)

In this case {r}O)} are computed as

r~O) = (r + r* )/3 = 2A /3;

rt(O) = [r exp(- i2n /3)+ r* exp(i2n /3 }}!3 = 2C / 3;

riO) = [r exp(i2n /3) + r* exp(- i2n /3)}!3 = 2B /3 (13.2.18)

and constant V is chosen so as to make {ro,rprz } nonnegative. The bias

V, evidently, may differ between different hologram sampies. The set of
these biases for hologram sampies may be optimized so as to improve the
quality ofreconstructed images ([14]).
When recording holograms with this method, one may encode the phase
,e
angle corresponding to unit vectors (e o , eno Z40 ) by means of the
mentioned above detour phase method by writing the components
{r0' r t,r 2} of the simplex-decomposed vector into groups of three
neighboring hologram resolution cells in the raster as it is shown in Fig.13-9
, a, band c. The latter two arrangements are more isotropic then the first one.
Co ordinate scale ratio for arrangement b) is 2:1.5 and for arrangement c) is
3Jj : 4 whereas for arrangement a) it is3: 1.
 558 Chapter 13

m=O m=1 m=2

00 1200 2400 \
I
n=O 00 1200 2400
J 2400 00 1200

n=1 00 1200 2400 I

1 1200 2400 00

00 1200 2400
n=2

-
00 1200 2400
J
a)
"
- b)

00 1200 2400 00 1200

2400 00 1200 2400

c)
00 1200 2400 00 1200

2400 00 1200 2400

Figure J3-9. Three methods of arrangement of recording medium resolution cells for
hologram encoding by decomposition of complex numbers with respect to a 2-D simplex.
Tripies of medium resolution cells used for recording one hologram sampie are outIined with
abold line.

For holograms recorded by this method according to the arrangement of

Fig. 13-8, a), images are reconstructed at angles 9~ = arccos(A./3~~) to the
axis ~ coinciding with the direction of the hologram raster rows and 9 11 =0
to the perpendicular axis. For the arrangements of Fig. 13-8, b) and c) the
reconstruction angles are, respectively, 9!; = arccos(A./3~~ ),
9 11 = arccos(2A./3) and 9!; = arccos(4A./3~~), 9 11 = arccos(2A./3).
One can show by using Eq.(13.2.14) that the following formula relating
values written in recording medium resolution cells with indices (m,n) with
sampIes {r(r, s)} of the mathematical hologram corresponds to the first
version of the coding method:

f m •n = 21- [(1 + sign{Re[r, .• exp(- i21Cm o /3)>»x

 13. COMPUTER GENERA TED HOLOGRAMS 559

{Re[r". exp(- i2rcm o /3 )]-IRe[r". exp(- i2rcm o /3 )]}+

(1- sign{Re[r". exp(- i2rcm o /3 )mx

{Re{r". exp[i2rc (mo + 1)/ 3]}+ /Re{r". exp[i2rc (mo + 1)/ 3m}l.
(13.2.19)
where horizontal and vertical indices {m,n} (Fig. 13-8, a» are counted as:
m = 3r + m o, m o = 0,1,2, n = sand Re(X) is real part of X .
The above described encoding methods based on additive representation
of complex vectors are applicable for recording on amplitude-only media,
both continuos tone and binary. In the latter case, projections of the complex
number on basic vectors are represented by varying the size of the
transparent aperture in each of the appropriate resolution cells.
For recording on phase media, additive representation of complex vectors
assumes complex vector representation as a sum of complex vectors of a
standard length. Two versions of such encoding are known: double phase
method and multiple phase method.
In the double-phase method ([ 10,15]), hologram sampIes are encoded as

r = Irl exp(i<p ) = Ao exp(i<p +) + Ao exp(i<p _}, (13.2.20)

where (<p I ,<p 1 ) are component phase angles defined by the following
equations:

(13.2.21)

that can be easily derived from the geometry of the method illustrated in Fig.
13-6, d.
Two neighboring medium resolution cells should be allocated for
representing two vector components. Formally, the encoded in this way
hologram may be written as

r m,n = Ao exp{ i ~ ',' + (_1}"'0 arccos(lr".1/2A o )]}, (13.2.22)

where {r ',' =Ir ".1 exp(i<p ',S )} are sampIes of the mathematical hologram,
indices of the recording medium resolution cells (m,n) are counted as
m = 2r + mo' m o = 0,1, n = s and Ao is found as half of maximal value of
{I r".I}·
560 Chapter 13

For hologram encoding with the double-phase method, images are

reconstructed in a direction normal to the hologram plane, because the
optical path difference of rays passing along this direction through the
neighboring resolution cells is zero. This holds, however, only for the central
area of the image through which the optical axis of the reconstruction system
passes. In the peripheral areas of the image, some phase shift between the
rays appears, thus leading to distortions in the peripheral image. This will be
discussed in more details in Sect. 13.3.
The method can also be used for recording on amplitude binary media
with phase-detour encoding of phases. Two versions of such an
implementation of the double-phase method were suggested ([16]). In the
first version, two elementary cells of the binary medium are allocated to each
of the two component vectors, their phases being coded by a shift of the
transparent (or completely reflecting) aperture along a direction
perpendicular to the line connecting cell centers as it is illustrated in Fig.13-
9, a. This technique exhibits above mentioned distortions due to mutual
spatial shift of elementary cells. The second version reduces these distortions
by means of decomposing each elementary cell into sub-cells altemating as
shown in Fig.l2-9, b. With the decomposition into K sub-cells, the relative
shift of elementary cells is K times less then without decomposition and the
peripheral image distortions are accordingly lower.

I...
1
~~~ 1 l.sl~):
27t 1 1 27t '"
1
1

~~

a) b)

Figure 13-10. Double-phase encoding method for hologram recording on a binaty phase
medium: two separate cells (a); decomposition of cells into alternating suh-cells
 13. COMPUTER GENERA TED HOLOGRAMS 561

The double-phase method is readily generalized to multi-phase encoding

by vector decomposition into an arbitrary number Q of equal-Iength vector
components:

r =L Ao exp(i<p q).
Q
(13.2.23)
q=1

As r is a complex number, Eq.(13.2.23) represents two equations for Q

unknown values of ~ q }. They have a unique solution defined by Eq.
13 .2.21 only for Q = 2. When Q > 2, ~ q} may be chosen in a rather
arbitrary manner. For example, for odd Q one may choose phases ~ q} so
as to make of them an arithmetic progression

<Pq+1 -<Pq =<Pq -<Pq-1 = Ll<p . (13.2.24)

In this case, the following equations can be derived for the increment Ll<p :

sin(QLl<p /2) Irl

(13.2.25)
sin(Ll<p /2) = Ao •

and for phase angles ~ q }:

<p q =<p + [q - (Q + 1)/2]L\<p, q =1,2,•••, Q . (13.2.26)

For odd Q, Eq.(13.2.25) boils down to an algebraic equation of

power (Q -1 )/2 with respect to sin 1 (Ll<p /2). Thus, for Q = 3,

(13.2.27)

For even Q, it is more expedient to separate all the component vectors

into two groups having the same phase angles <p + and <p _ that are defined
by analogy with Eq.(13.2.21) as

<p + =<p + arccos~rl/QAo ); <p _ =<p - arccos~rl/QAo ). (13.2.28)

562 Chapter 13

With multi-phase encoding, the dynamic range of possible hologram

values may be extended because the maximal reproducible amplitude in this
case is QA o ' Most interesting of the Q > 2 cases are those with Q = 3 and
Q = 4 because the two-dimensional spatial degrees of freedom of the
medium and hologram recorder may be used more efficiently through
allocation ofthe component vectors according to Figs.l3-8, c) and 13-9, b)
and c).
All hologram encoding methods can be treated in a unified way as
methods that explicitly or implicitly introduce to recorded holograms some
form of a spatial carrier similarly to how optical holograms are recorded.
This can be shown by writing Eqs.13.2.5, 13.2.8, 13.2.9, 13.2.19 and 13.2.22
in the following equivalent form explicitly containing hologram sampies
multiplied by those of the spatial carrier with respect to one or both
coordinates:

[m,n = Re{rr,s exp(- i2rcm / 2)}/4 + b,

m=2r+m o ; m o =0,1; n=s; (13.2.5')

[m,n = ~ retf {Re[rr,s exp(- i2rcm / 2) ]},

m=4r+2m OI +m 02 ; m 01 ,m 02 =0,1; n=s; (13.2.8')

[m,n = ~retf{Re[rr,s exp(- i2rc (m + 2n)/ 4)]},

m=2r+m o; n=2s+no,m o,no =0,1; (13.2.9')

[m,n = ~ ~ (hlim{Re[rr,s exp(- i2rc (m + p)/ 3) ]})x

retf{Re[rr,s exp(- i2rc (m + p + 1/2)/3)]};

m=3r+m o;m o =0,1,3; n=s; (13.2.19')

[m,n = Ao exp{i ~ r,s - eos(2rcm / 2)areeos(lrr,s 1/2Ao )]};

m=2r+m o;m o =0,1; n=s, (13.2.22')

where retf(X) is the "rectifier" function

X, X 2 0
retf(X)= { , (13.2.29)
0, X <0
 13. COMPUTER GENERA TED HOLOGRAMS 563

and hlim(X) is the "hard-limiter" function:

I, X ~O
hlim(X) = { . (13.2.30)
0, X <0

As one may see from these expressions, the spatial carrier has aperiod no
greater than one half that of hologram sampling. At least two sampies of the
spatial carrier have to correspond to one hologram sampie in order to enable
reconstruction of amplitude and phase of each hologram sampie through the
modulated signal of the spatial carrier. This redundancy implies that, in order
to modulate the spatial carrier by a hologram, one or more intermediate
sampies are required between the basic ones. They may be determined by
any of variety of interpolation methods.
The simplest interpolation method, zero order interpolation, simply.
repeats sampies. It is this interpolation that is implied in the recording
methods described above. For instance, according to Eq.(13.2.5') each
hologram sampie is repeated twice for two sampies of the spatial carrier, in
Eq.(13.2.8') it is repeated four times for four sampies, etc. Of course such an
interpolation found in all modifications of the detour phase method yields a
very rough approximation of hologram intermediate sampies. In Sect.l3.3 it
will be shown that the distortions of the reconstructed image caused by
improper interpolation of hologram sampIes manifest themselves in aliasing
effects.
As we have shown in eh. 9 aliasing free interpolation can be achieved
with discrete sinc-interpolation. Discrete sinc-interpolated values of the
desired intermediate sampies can be obtained by using, at the hologram
synthesis, Shifted DFT with appropriately found shift parameters that
correspond to the position of required intermediate sampies of the hologram.
It should be also noted that the symmetrization method may be regarded as
an analog of the method of Eq.(l3.2.5) with discrete sinc-interpolation of
intermediate sampies. In the symmetrization method, such an interpolation is
secured automatically and, as will be seen in Sect.13.5, the restored images
are free of aliasing images.
Along with methods that introduce the spatial carriers implicitly, explicit
introduction of spatial carriers is also possible that directly simulates optical
recording ofholograms and interferograms.
Of the methods oriented to amplitude-only media, one can mention those
of Burch ([ 17]):

(.n = 1+ Ir Icos(2njm +
m,n <j> m,n ) (13.2.31)

and ofHuang and Prasada ([18])

564 Chapter 13

r m,n = Irm,n 1[1 + cos(2njm + ep m,n ) ] (13,2.32)

where j is frequency of the spatial carrier,

Of binary-media-oriented methods, one may mention that described in
the overview [5]:

r m,n = hlim ~os [ arcsin~rm,n 1/A o) ] + cos(2njm + ep m,n )}, (13.2.33)

where A o is maximal value of Irm,n I·

Of the phase-media-oriented methods, we may mention that of Kirk and
Jones ([19]):

r m,n = A o exp{i [ep m,n - hm,n cos(2njm) ]}

where hm,n depends in a certain way on Ir I and on the diffraction order

m,n

where the reconstructed image should be obtained. In asense, this method is

equivalent to the multi phase encoding method. When j = 1/2 and

(13.2.34)

it coincides with the double phase encoding method according to

Eq.(l3.2.22).
 J 3. COMPUTER GENERA TED HOLOGRAMS 565

13.3 RECONSTRUCTION OF COMPUTER GENERATED

HOLOGRAMS

In the reconstruction stage, computer generated holograms synthesized

with the use of discrete representations of wave propagation transformations
are subjected, to analog optical transformations. The discrepancy between
them and their discrete representation affects in a certain way the hologram
reconstruction result. Transformations of a digital signal into a physical
hologram according to the method of hologram recording its influence as
weIl. In order to illustrate techniques for analysis of the hologram
reconstruction that account for these factors, we will consider in this section
the reconstruction, in an analog set-up performing the optical Fourier
transform, of computer generated Fourier holograms for three methods
of hologram encoding: for symmetrization method, for orthogonal
encoding method (Eq.(13.2.5», and for double phase recording on a phase
medium (Eq.(l3.2.22».
The following characteristics of the hologram recording device affect the
reconstruction result: type of the discretization raster, discretization intervals,
recording aperture and physical size of recorded holograms. Let (;-,17) be
physical coordinates on the hologram recording medium, (~;-, ~ 17) be
discretization intervals of the rectangular sampling grid along coordinates
(;,17), (;0,170) be shift parameters that depend on the geometry of
positioning the hologram in the reconstruction set up, h rec (;,17) be recording
aperture function and w(;, 17) be a window function that defines physical
size of the recorded hologram: W(;-,17)~O, when (;-,17) belongs to the
hologram area and W(;-,17) = 0, otherwise (see Fig. 13-11).

Figure 13-11. Definitions related to recorded physical computer generated holograms

566 Chapter 13

For the symmetrization method, the following equation describes

conversion of the numerical matrix of the mathematical hologram {r".} into
the physical hologram r(~, T)) recorded on an amplitude-only medium:

, .
where b is a constant bias required for eliminating negative values in
recording {r".} and indices (r, s) run over all available hologram sampIes.
Eq. (13.3.1) may be rewritten in a form ofthe convolution:

r(~ ,T)) = w(~ ,T))' {hree(~ ,T))®

,t~(r,.. +b~(~ +~o -rL1(~5(T) +T)o -SL1T))}, (13.3.2)

where ® stands for the convolution and summation limits are changed to
[- 00,00] having in mind that hologram window function w(~ ,T)) selects
available hologram sampIes from virtual infinite number of sampIes.
Let now (x,y) be coordinates in reconstructed image plane situated at a
distance D from the hologram plane and A. be wavelength of the
reconstructing light beam. According to the convolution theorem of the
integral Fourier transform, the result of the optical Fourier transform
performed at the reconstruction of the hologram r(~ ,T)) can then be
represented as:
0000

Aree(x,y)= J Jr(~,T))exp[i2n(x~ + YTJ)/W~dT) =

-00-00

W(x,y)®{ H,ee(x,y )_!Lt.t (r". +b~ (~ +~o - rL1~)3 (T) +T)o - SL1T))x

exp[i2n (x~ +YTJ )/W ]d~dT)} =

W(x, y)® {Hree (x, y)exp[- i2n (x~o +YTJo )/W]x

L, L. (r". +b)exp[- i2n (rL1~x +sL1T)Y )/W] , (13.3.3)

 13. COMPUTER GENERA TED HOLOGRAMS 567

where

f fW(~,ll)exp[i21t(X~ + Y1l)/A.D~d1l
00 00

W(x,y)= (13.3.4)

and

f fh,j~,ll)exp[i21t(X~ + Y1l)/W~dll
00 00

H,ec(x,y)= (13.3.5)

are Fourier transforms of the hologram window function and of the aperture
function of the hologram recording device.
Introduce now an array A}~), k = O,...,2NJ -1, 1== O,I,••. ,N2 -1, of
sampies of the object wave front. In the symmetrization method, the array is
symmetrical as defined by (Eq. 13.2.1). For Fourier holograms, mathematical
hologram r"s is computed as Shifted OFT ofthis array:

r rs oc 2~J~JA(o)
L L k,l exp {"21t [(k+u)(r+ p)+ (I+V)(S+q)]} ,
I (13.3.6)
, k~O I~O 2N J N2

where (u, v) and (p,q) are shift parameters. Substitute Eq. 13.3.6 ioto Eq.
13.3.3 aod obtaio:

exp[- i21t (r~~x + s~1lY )/ A.D ], (13.3.7)

or, using Poisson's summation formula (Eq. 3.3.16),

568 Chapter 13

(13.3.8)

As may be seen from Eq.(13.3.8), at synthesis and recording of Fourier

holograms one can take p = q = O. One can also neglect the phase term
exp[- i27t (x~ 0 + J'l1 0 )/f,J)] that is irrelevant for display holograms. With
these settings, obtain finally:
Arec{x,y)oc

H rec ( x,y'
) ~
{ m~-oo ~2~1~IA-(O)W[k+U
L. L. L L k./
ß~ l+v ß1lY
-----+m,-----+n
)
n~-oo k~O /~O 2N f,J)
I N 'AD 2

+b f f w[k2N+
m~-oon~-oo
U _
I
ß~ x+ m, I + v_ßll Y + n)}.
f,J) N2 f,J)
(13.3.9)

It follows from this formula that

• the hologram placed into the optical Fourier system reconstructs the
original object wave front in several diffraction orders defined by
indices m and n;
• the object wave front is reconstructed by interpolation of its sampies
with the interpolation function equal to Fourier transform of the
hologram window function;
• the pattern of the reconstructed images sampIes in the diffraction
orders is masked by a function H rec (x, y ), spatial frequency
response of the hologram recording device.
• constant bias in hologram recording results in appearance in the
reconstructed image of bright spots in the center of each diffraction
order (last term in Eq. 13.3.9)
Arrangement of diffraction orders in the reconstructed image plane under
shift parameters u = -NI' V = -N2 /2 is shown in Fig. 13-12.
In case of orthogonal encoding according to Eq.(l3.2.5), one obtains,
in the same denotations that were adopted above:
 13. COMPUTER GENERA TED HOL OGRA MS 569

(13.3.10)

., IU . . . . . . . . . . .
.
1

b)

a)

Figure 13-12. Reconstruction of a hologram synthesized using image symmetrization by

duplication (a) for a test object shown in b).

Following the reasoning and settings used in deriving Eq.(13.3.8), one

can obtain that, in the Fourier transform reconstruction scheme, such a
hologram reconstructs the following wave front:
 570 Chapter 13

A~(X'Y)OCH~(X,+oH~+ ";;)] x

f fffA(O)W(k+U-Ll~X+m+!
m:-oo n:-ook:O /:0 k ,/ NI ').J)
l+v _Ll11Y +n)+
2' N 2 ').J)

~ .t.....t.....t....
.t.... ~ ~ ~A(O) N,-kN 2 -/
'w(k- -+- - -Ll~+x m + - ,I-+-v- - - +Y n) +
U 1 Llll
m:-oo n:-ook:O / : 0 ' NI AD 2 N2 ').J)

b COS(1t Ll~ADX) f f w( kNI+ + Ll~ x+ m+ !,2 IN+ v+ Ll11 Y + n) },

m:-oo no-oo
U
').J) 2 ').J)

(13.3.11)
where, as above, (u, v) and (p,q) are shift parameters of SOFT used in
computing the mathematical hologram, NI and N 2 are dimensions of the
array {A1~)}. As in the above case of hologram encoding by symmetrization,
one should take p = q = 0 and ~o =110 = 0 at hologram synthesis and
recording.
One can see from Eq. (13.3.11) that, similarly to the above case, the
reconstructed image contains a number of diffraction orders, masked by the
function H,Jx,y), spatial frequency response of the hologram recording
device, and a central spot in several diffraction orders due to the constant
bias in the hologram. But here, in contrast to the above ca se, each
diffraction order contains two superimposed images of the object, - a direct
image and its conjugate rotated by 1800 with respect the former. Each of
them is additionally masked by functions

{cos[.(~+ ";;)] and {S+(~- ";;)], respectively. Therefore, in

the central region of the direct image the conjugate image is suppressed, but
at its periphery the aliasing conjugate image has an intensity comparable
with that of the direct image. This is accounted for by the fact, that here
additional hologram sampIes necessary for representing the spatial carrier
are obtained by zero order interpolation of sampIes of the mathematical
hologram.
 13. COMPUTER GENERA TED HOLOGRAMS 571

The pattern of diffraction orders of the direct and conjugate images and
their corresponding masking functions are shown in Fig.ll-13 for
u = -NI /2 and v = - N 2 /2 . One may easily see the direct image and its
conjugate, an aliasing image whose contrast increases along the horizontal
axis from the center to the periphery.

COS[J!(.!.4 + 6,;x
AD

Figure 13-13. Image reconstruction for the orthogonal encoding method. At the bottom,
spatial weighting functions ofthe direct and conjugate images are depicted.

The phase recording of the hologram on phase-only media according

Eq.(13.2.22) is described, in the same denotations, as

r(~'17)= W(~'17~,~s~toexP~[qJ"s + (-tto arccos~r"sI/2Ao)nx

h,J~ + ~o - (2r + mo),1.~'17 + 170 - sA17]=
572 Chapter 13

(13.3.12)

In a Fourier hologram reconstruction setup, this hologram is Fourier

transformed and reconstructs the wave front described as

f ff(~ ,Tl )exp[- i21t (x~ + Yll )/Al>]d~dTl =

00 00

Aree (x, y) =

W(x,Y)<8> [ H ~(x'{~:J':J:xP~[q> . . + (-1)" .rccos~r. . 1/2A. ~lx

exp(- i21t {[(2r + mo)Ll~ - ~o]x + (SLlTl -Tl o)Y}/Al» ]=

W(x,y)® {2H ree (X,y )exp(- i21t {[(~o - Ll~ /2)x +TloY ]}/Al» x

lc+ A;; lttJ. . 1/2A, )exp(iq>...lex{-i2n (ZrAE,x.; sA1]Y)] +

sm
o ( Ll~ XJ L..~ L..~
1t - -
1- Irr,s 1 exp(0lCJ> r s )exp[- I021t
--2
2
(2rLl~x + SLlTly)]l .
Al> r~-oos~-oo 4A o ' "AD

(13.3.13)

By substituting rr,s with its expression through SDFT( u, V; p, q ):

Irr,S IexpvCJ>
(o
r,S
)= ~1~IA(o)
L.. L.. exp{02
I 1t
[(k+uXr+p) + (I + vXs +q)]}
k,1
k~O I~O NI N2
(13.3.14)

and introducing an auxiliary function Ai1 defined through equation

 13. COMPUTER GENERA TED HOLOGRAMS 573

(13.3.15)

obtain after some transformations and setting p = q = 0:

{ cos ( 7t
Ll~X)~L.J ~~~A(O)W(k+U
-- L.JLL lLl~X
----- +m,- - - -LlllY
I+v kJ- + n) x
')J) m~-oo ,,~-ook~O '~O NI ')J) N1 ')J)

Ll~X)~ ~~~A-(O)W(k+U
• ( 7t - - L.J L.JLL
sm lLl~X
----- +m,- - - -LlllY
I+v - + n )}
kJ
')J) m~-oo "~-,,,k~O '~O NI ')J) N1 ')J)

(13.3.16)

The result of reconstruction of a hologram recorded on a phase medium

by the two phase method is, thus, similar to the reconstruction of an
hologram with orthogonal encoding (see Eq. 13.3.11). Image is also
reconstructed in a number of diffraction orders masked by the fimction
H,.,c (x, y). There is also superposition of the aliasing image described by
the fimction Ä!:J over the original image A1~) and the original and aliasing

images are additionally masked by the fimctions cos(7t Ll~X) and

sin( 7t Ll~X ) , respectively. In the center of the proper image aliasing image
is fully attenuated but over the peripheral area it may be of the same intensity
as the proper image. In contrast to the orthogonal coding method, the aliasing
image here is not conjugate to the original one, but is similar to it, in asense,
because, according to Eq. 13.3.15, it has the same phase spectrum and a
distorted amplitude spectrum. Unlike the holograms coded via the
symmetrization or orthogonal methods, double- phase encoding does not
produce a central spot in the diffraction orders of the reconstructed image
because the hologram is recorded in the phase medium without an amplitude
574 Chapter 13

bias. Fig.11-14 shows the pattern of diffraction orders in the reconstructed

image plane for this case with u = -NI /2 and v = - N 2 /2 .

Figure J3- J4. Hologram reconstruction for double phase recording method: a) -arrangement
of diffraction orders (Solid and dashed arrows indicate the basic and aliasing images with
diffraction order shown in boxes; spatial masking functions for both images are depicted at
bottom correspondingly by solid and dashed Iines); b) - reconstructed image for the test object
ofFig. 13-12, c).
 13. COMPUTER GENERA TED HOLOGRAMS 575

References

1. L. Yaroslavskii, N. Merzlyakov, Methods of Digital Holography, Consultance Bureau,

N.Y.,1980
2. H. I. Bjelkhagen, Silver Halid Recording Materials, Springer Verlag, Heidelberg, 1993
3. RS. Nesbitt, S. L. Smith, RA. Molnar, S. A. Benton, Holographic recording using a
digital micromirror device, Conf. on Practical Holography, Proc. SPIE, 3637, 12-20,
1999
4. T. Kreis, P. Aswendt, R Hoeffiing, Holographic reconstruction using a digital
micromirror device, Optical Engineering, v. 40 (6), pp. 926-933, June 2001
5. W.H. Lee, Computer generated holograms: techniques and applications, In: Progress in
Optics, E. Wolf, ed., v. XVI, North Holland, 1978, pp. 121-231
6. W. I. Dallas, Computer generated holograms, in: The Computer in Optical Research,
Methods and Applications, B. R. Friede, Ed., Topics in Applied Physics, v. 41, Springer,
Berlin, 1980, pp. 297-367
7. O. Bryngdahl, F. Wyrowski, Digital holography - computer-generated holograms, in:
Progress in Optics, E. Wolf., ed.,v. XXVIII,EIsevier Science Publishers B.V., 1990, pp.
3-87
8. L. B. Lesern, P. M. Hirsch, J. A. Jordan, Kinoform, IBM Joum. Res. Dev., 13, 150, 1969
9. L. B. Lesern, P. M. Hirsch, 1. A. Jordan, Computer synthesis of holograms for 3-D
display, Commun. Of Associat. For Computing Machinery, v. 11,1968, pp. 661-674
10. B. R. Brown, A. Lohmann, Complex spatial filtering woth binary masks, Appl. Optics, 5,
No. 6, 967-969 (1966)
11. B. R. Brown, A. Lohmann, Computer generated binary holograms, 1MB J. Res. Dev., v.
13, No. 2,160-168 (1969)
12. H. P. Herzig, Ed., Micro-optics - Elements, Systems, and Applications, Taylor&Francis,
London,1997
13. C. B. Burckhardt, A simplification of Lee's method of generating holograms by
computer, Appl. Opt., v. 9, No.8, p. 1949 (1970)
14. P. Chavel, J. P. Hugonin, High quality computer generated holograms: the problem of
phase representation, JOSA, v. 66, p. 989 (1976)
15. 1. W. Goodman, D. C. Chu, I. R. Fienup, Recent developments in computer holograms,
Proc. SPIE, v. 41, pp. 155-159, 1974
16. C. K. Hsueh, A. A. Sawchuk, Computer generated double phase holograms" Appl. Opt.,
v. 17, No. 24, pp. 3874-3883, (1978)
17. 1. J. Burch, Aigorithms for computer synthesis of spatial filters, Proc. IEEE, v. 55,
999,1967
18. T. S. Huang, E. Prasada, Considerations on the generation and processing of holograms
by digital computers, MITIRLE Quat. Progr. Rept., 1966, v. 81
19. 1. P. Kirk, A. L. Jones, Phase-only complex valued spatial filter, JOSA, v. 61, No. 8, pp.
1023-1028, 1971
INDEX

band limited functions, 84. See

band limited functions
"cutting" matrices, 264 bandwidth limitations, 40
"reflected" cutting matrices, 266 basis functions, 17, 18, 19, 65
"shadow" matrix, 266 Bayes approach, 414, 415
4-F correlator, 419 binary media, 554, 557, 558, 566
aposteriori probability, 414, 447 Binary-to-Gray Code
apriori probability, 414, 415, 418 Permutation, 252
ABCD-transform, 52 biorthogonal basis, 559, 561, 562
Abel transform, 60 bit reversal. See bit reversal
additive noise, 67, 68, 69, 70 bit reversal permutation, 250
additive signal independent noise Bit Reversal Permutation, 250
model,67 border processing, 223, 224
aliasing effects, 120, 121, 122 called image reconstruction, 131
alpha-trimmed mean, 279 camera-obscura, 27
amplitude-only media, 554, 555, cardinality, 366
556,558,565,569 Cardnl-jilter, 530, 531
analog signals, 12, 13 cascade jiltering, 213, 214, 215,
analytical signal, 53, 55, 56 226
Andrey N. Kolmogorov, 317 centrallimit theorem, 68, 73
anomalous errors, 423, 438, 442, chirp-function, 51, 52
444, 445, 456, 459, 460, 461, chirp-functions, 49
462,463 Coded aperture, 131
aperture correction, 323, 325, co ding co-ordinates of rare
327 symbols, 151
autocorrelation function, 280, co ding ofrare symbols, 150
281,287 Combined algorithms, 231, 232
 578

combined amplitude/phase direct binary-to-Gray code

media, 554, 555 permutation, 253, 254
compressor-expander Direct matrix sum, 245
(compander),135 Discrete Cosine Transform, 189,
computed tomography, 131 190
consistency principle, 161 discrete jiltered back projection
continuous signals, 12, 13, 16, image algorithm, 405
24,25 Discrete Fourier Transforms,
convolution based interpolation, 170,173,175,201
85 discrete frequency response, 169
convolution basis functions, 85 Discrete Fresnel Transform, 199,
convolution integral, 29, 30, 43, 201,203,204
54,61,64 discrete impulse response, 164
convolution theorem, 84 discrete point spread function,
Convolution theorem, 43, 179 164, 168
correlationfunction, 68,69, 73 discrete PSF, 167, 169,201
correlation interval, 442 discrete ramp-jilter, 403
correlational accumulation. See discrete sampling theorem, 173,
correlational accumulation 176, 186, 189
correlational averaging. See Discrete sampling theorem, 185
corre1ational averaging Discrete signals, 12, 15
cross-correlation function, 281 discrete sinc-function, 176, 188.
cumulative distribution See discrete sinc-function
histogram, 275 discrete sinc-interpolation, 377,
cyclic convolution, 180, 185 382, 384, 385, 386, 387, 388,
data fusion , 342 389, 390, 391, 392, 393, 394,
dataredundancy, 148 397,403,406,412
dc-component, 182, 191, 193 Discrete Sine Transform, 194
DcST, 192, 193,260,262,263 discretization interval, 81, 107,
DCT, 189, 190, 191, 192, 193, 113
194,195,260,262,263,274 discretization with gaps, 117
DCT based discrete sinc- discretization with sub band
interpolation, 389 decomposition, 117
delta-function, 22, 25, 26, 69 distribution histogram, 275, 276,
detour phase method, 557, 559, 290
563,569 dot product, 18
DFrT,201 double-phase method. See
digitaljilter, 164, 167, 168, 169 double-phase method
digital jiltering, 167, 169, 172, DPCM. See DPCM
201 DPCM with feedback, 155, 156,
digital signals, 12, 13, 24 159
DST, 194, 195
579

dyadie shijt, 90 Fresnel integral. See Fresnel

edge deteetion, 537, 541, 542, integral
543,544 frined-funetion, 205, 206, 207
empirieal Wiener jilter, 319, 320, frine-funetion, 50
323,325 Gabor transform, 64
energy eompaetion property, 94, general digitization, 13, 147, 149
157 Gibb's effeets, 40, 88
equivalenee eells, 13 global jiltering, 328
false eontours, 139 global histogram, 276
far zone diffraetion, 35 Haar funetions, 93, 94
Fast Haar Transform, 255 Haar Transform, 254
fast Hadamard transform, 250 Hadamard Permutation, 252
Fast Walsh Transform, 254 Hadamard permutation matrix.
fast Walsh-Paley transform, 251 See Hadamard permutation
FFT method for generating matrix
eorrelated numbers, 297 Hadamard produet, 226
jiltered baek projeetion method, hard thresholding, 320
58. See filtered back projection Hartley transform, 39, 45
method histogram equalization, 141,354,
jiltering in signal domain, 211 366,367,368
jiltering in the DFT domain, 226 histogram matehing, 353, 354
jiltering in transform domain, hologram eneoding, 547, 555,
226,227 561,564,566,568,576
foeallength,35 homogeneous, 131, 133, 134
Foeal Plane Invariant Diserete homogeneous transformation, 24
Fresnel Transform, 202, 203 homogeneous transformations,
Foeal Plane Invariant Partial 162
Diserete Fresnel Transform , Horizontal matrix sum, 245
203 Hotelling expansion, 101
Fourier holograms, 552, 553, ideallow passjilters, 169, 189
556,573,574 image deblurring, 323, 325
Fourier method. See Fourier image homogenization, 485, 486
reconstruction method image reeonstruetion, 314, 355,
Fourier reeonstruetion method, 360
58 image restoration, 314, 315, 324,
Fourier speetrum, 39, 41, 42, 49 329,331,343,356,358,363
Fraetional Fourier transform, 52 Image segmentation, 541, 542
Fraunhofer approximation, 35 Impulse noise model, 70
frequeney representation, 331 impulse response, 26, 43
frequeney response. See inhomogeneous quantization,
frequency response 143, 146
Fresnel holograms, 551, 552
 580

inhomogeneous transformation, loss-less coding, 150, 159

24 lossyeompression, 150
inner product, 18 magnetie resonanee imaging,
integral Cosine transform, 44, 45 131
integral Fourier transforms, 37 MAP-estimate, 414, 417
integral Fresnel transform, 37, matehed filter, 416, 418, 419,
49 421, 422, 424, 430, 438, 439,
integral Kirchhoftransform, 37 441, 442, 448, 459, 460, 461,
integral transform, 22, 26, 39, 63 463,464
inverseJilter, 325, 327, 356, 357 matehedfiltering, 418, 419, 463
inverse Fourier transform, 40, mathematieal hologram, 547,
50,52,54,58,64 554, 555, 557, 559, 560, 561,
Inverse matrix, 244 564,565,572,573,576
inverse problem, 314 Matrix produet, 244
inverse problems, 30 Matrix sum, 244
jinc-junction, 47, 60 Matrix transposition, 245, 252
Karhunen-Loeve bases, 99 Max-Lloyd quantization, 136
kinoform, 146,300, 302, 555 MDCT, 195, 197, 198
Kirchhoff's integral, 34, 35 mean squared error, 315
Kronecker delta, 19, 22 mean value, 278, 279, 283, 311
Kronecker matrix, 246, 249, 251, median, 279
253, 254, 255, 259. See Mellin transform, 39,48,49
Kronecker matrix ML-estimates. See ML-estimate
Kronecker product, 246 MLT, 195, 196, 197
k-th law nonlinearity, 498, 500 Modijied DCT, 195
Laplacian, 337, 345, 492, 509 Modulated Lapped Transform,
Laplacian operator, 337 195
Layered Kronecker matrix, 246 modulation transfer funetion, 43
linear transformations. See moire effects, 121
linear transformations moiri noise, 69
loeal adaptive eorrelators, 482 Monte Carlo methods, 305
loeal eontrast enhancement. See Monte-Carlo simulation, 291
local contrast enhancement MSE-optimal filtering, 315
loeal criteria, 16 multi-phase encoding, 567, 568
local filtering, 328, 329 Multiplieative noise, 69
loeal histogram equalization, multivariable distribution
365, 525, 530, 532, 538, 539, histogram, 276
540 mutiresolution, 61
loeal histograms, 276, 277 mutual correspondenee
loeal mean, 213, 218. See local principle, 161, 165
mean NBH2-histogram equalization,
loeal spectrum, 63 531,532
581

N-dimensionallinear space, 18 power spectrum, 68, 69, 281, 282,

near zone diffraction, 34, 49 288, 289, 294, 297, 299, 300,
neighborhood. See neighborhood 301,303
Norbert Wiener, 317 primary pseudo-random number
normal distribution, 68, 70, 74 generator, 292, 297
normal errors, 423, 424, 429, principal components, 101
436,438,444,459,464 probability density, 16,68,69, 74
on-axis holograms, 555 probability distribution density,
one-dimensional histogram, 276 275
optimal adaptive correlator, 473, programmed diffuser holograms,
475, 477, 482, 485, 487, 488, 300,303
489, 492, 496, 497, 499, 500, projection theorem, 58, 59
502,504 PRUNED ALGORITHMS, 264
optimalfilter, 440, 446, 447, 448, pruned FFT, 264
450,451,459 pruning matrices, 264, 266
optimal scalar quantizer, 135 pruning matrix, 264
order statistics, 275, 278, 279, pseudo random sequences, 291
280 pseudo-random diffuser, 145,
orthogonal encoding, 559, 577, 146
579 P-th law quantization, 143, 144,
orthogonal functions, 19 145
orthogonal matrix, 21 pyramid coding, 156
orthonormal basis, 19, 20 QDFT, 269, 271, 272, 273
parallel filtering, 213, 214, 215, quantization intervals . See
216,217,218,219,226 quantization intervals
paraxial approximation, 35, 38 quantization levels, 12,24
Parseval's relationship, 20 Quantized DFT, 271
Partial Discrete Shifted Fresnel Quantized Discrete Fourier
Transform, 202 Transform, 269
partial Fresnel transform, 51 Quantized signals, 12
pepper and salt noise, 70 quantized value, 133, 138
Permutation theorem, 178 Rademacher functions, 89, 90,
phase-only filter 93
phase-only filter, 494 radix-2 FFT, 240
phase-only media, 554, 555, 577 Radon transform, 57, 58, 60
P-histogram equalization, 365, ramp filter, 59
366,368,530,531,538,539 random processes, 16
point spread function, 26, 28, 33, rank, 279, 280
35,67,72,73,75 raster points, 81
point-wise transformations, 24 Rayleigh's criterion, 29
Poisson distribution, 69, 70 reconstruction aperture
Poisson noise model, 69 functions, 80
 582

rectangular impulse funetions, signal diseretization, 79, 80, 81,

81 94, 107, 111, 126, 127, 133
reet-funetion, 40, 47,50 signal elements, 12
reeursivefiltering, 211, 212, 213 signal p-shift. See signal p-shift
regularization, 325, 355 signal reeonstruetion, 13
rejeeting filter, 320, 322, 325, signalreeove~, 314
331 signal sampies, 12, 16
representative signal, 13 signal spaee, 13, 15, 16, 17, 18,
resolving power, 29, 30, 72, 75, 24
76 signal spaee metries, 14
run length eoding, 151 signal spaee-bandwidth produet,
sampling, 106 113
sampling points, 81 signal statistieal eharaeteristies,
sampling theorem, 85, 106, 112, 16
120 signal transform, 21, 53
sealar (element wise) signal varianee, 278
quantization, 79 signal zooming, 376,384,397
sealar filter, 164 signal's eomplex amplitude, 55
scalar filtering, 316, 330, 342 signal's envelope, 55
sealar produet, 18, 19,20,21,23 signal's representation, 18
sealing basis funetions, 85 signal-to-clutter ratio, 472, 475,
Sealing theorem, 42 481,488,492,500,504
SDFrT, 201, 202, 205 signal-to-noise ratio, 318, 320,
See "decimation-in-frequency" 324
FFTFT. sine-funetion, 40, 47,50,60
See "decimation-in-time" FFT sine-funetions, 82
algorithm. singular value deeomposition,
self eonjugate matrix, 21 102
se~eonjugate,23 SizeEV-eontrolled Sigma-filter,
sensor aperture funetions, 80 530
sequeney, 91 sliding window transforms, 63,
shift basisfunetions, 81, 85, 107, 65
113 soft thresholding, 320
shift theorem, 84 spaee-frequeney representation,
Shifi theorem, 41, 178 64
shifted eyclie eonvolution, 180 spatiallight modulator, 37, 38
Shifted Diserete Fourier speekle eontrast, 75
Transforms, 173 speekle noise, 67, 72, 75, 76
Shifted Diserete Fresnel speetral density, 68, 69
Transform, 201 speetrum, 12, 22, 25, 26, 41, 42,
signal digitization, 13 49, 51, 53, 58, 59, 63, 64, 65,
69
583

splines, 215 whitening filter, 446, 448, 457

standard deviation, 278, 285, Wiener filters, 316, 317, 319,
301,307,311 320,329,330
stationary proeess, 68 windowed transforms, 63
statistical (entropy) eoding, 150 zonal quantization, 143, 159.
strobe effeets, 120 zooming, 376
sub-band deeomposition co ding,
156
super-resolution, 343
symmetrization, 86
symmetrization method, 556,
569,571,572,573
thermal noise, 68
three pass rotation algorithm,
400
time-jrequeney representation,
64,65
transfer funetion, 24
transform kernei, 22, 25, 62, 63
transform matrix, 21
transversal filtering, 211
two-dimensional simplex, 562
uneorrelated proeess, 69, 72, 73
unitary transform, 21
Unsharp masking, 369, 370
variable length coding, 150
variation al row, 278, 279, 285
vector filter, 164
Vertical matrix sum, 245
Walshfunctions, 89,90,91,94
Walsh Transform, 252
Walsh-Hadamard transform, 249,
250,251
Walsh-Paley Transform, 250, 251
wavelet coding, 156
wavelet shrinkage, 320, 335, 337,
340,373
wavelet transform, 61, 62
wavelet transforms, 61, 62, 64, 65
Weber-Fechner's law, 139
weighted histograms, 276
white noise, 69