Digital Image Processing I

Roger L. Easton, Jr.

9 September 2010
Contents

Preface ix

1 Basic Principles of Digital Image Processing 1

1.1 Digital Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2 Digitization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Review of Sampling 7
2.0.1 Ideal Sampling of 1-D function . . . . . . . . . . . . . . . . . 8
2.1 Aliasing –Whittaker-Shannon Sampling Theorem . . . . . . . . . . . 11
2.2 Realistic Sampling –Averaging by the Detector . . . . . . . . . . . . 12

3 Review of Quantization 21
3.1 Quantization: A/D Conversion . . . . . . . . . . . . . . . . . . . . . 21
3.1.1 Tone Transfer Function . . . . . . . . . . . . . . . . . . . . . . 21
3.2 Quantization Error (“Noise”) . . . . . . . . . . . . . . . . . . . . . . 25
3.2.1 Signal-to-Noise Ratio . . . . . . . . . . . . . . . . . . . . . . . 28
3.2.2 Example: Variance of a Sinusoid . . . . . . . . . . . . . . . . . 29
3.2.3 Example: Variance of a Square Wave: . . . . . . . . . . . . . . 30
3.2.4 Variance of “Noise”from a Gaussian Distribution . . . . . . . 30
3.2.5 Approximations to SNR . . . . . . . . . . . . . . . . . . . . . 30
3.2.6 SNR of Quantization . . . . . . . . . . . . . . . . . . . . . . . 32
3.3 Quantization to Few Bits . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.3.1 Improved Gray Scale (IGS) Quantization . . . . . . . . . . . . 35
3.3.2 Quantizers with Memory –Error Di¤usion . . . . . . . . . . . 35

4 Point Operations 39
4.1 Geometrical Operations . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.2 Least-Squares Solution for Warping . . . . . . . . . . . . . . . . . . . 42
4.3 Common Geometrical Operations . . . . . . . . . . . . . . . . . . . . 45
4.4 Pixel Transfers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.5 Pixel Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.6 Point Operations on Single Images . . . . . . . . . . . . . . . . . . . 46
4.6.1 Image Histograms . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.6.2 Histograms of Typical Images . . . . . . . . . . . . . . . . . . 48
4.6.3 Cumalative Histogram . . . . . . . . . . . . . . . . . . . . . . 48

v
vi CONTENTS

4.6.4 Histogram Modi…cation for Image Enhancement . . . . . . . . 49

4.6.5 Jones Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.7 Histogram Equalization (“Flattening”) . . . . . . . . . . . . . . . . . 51
4.7.1 Example of Histogram Equalization –1-D “Image” . . . . . . 53
4.7.2 Nonlinear Nature of Histogram Equalization . . . . . . . . . . 53
4.8 Histogram Speci…cation . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.9 Application of Histograms to Tone-Transfer Correction . . . . . . . . 55
4.10 Application of Histograms to Image Segmentation . . . . . . . . . . . 56

5 Local Operations 57
5.1 Window Operators –Correlation . . . . . . . . . . . . . . . . . . . . 57
5.2 Convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.2.1 Convolutions –Edges of the Image . . . . . . . . . . . . . . . 63
5.2.2 Convolutions –Computational Intensity . . . . . . . . . . . . 63
5.2.3 Smoothing Kernels –Lowpass Filtering . . . . . . . . . . . . . 64
5.2.4 Di¤erencing Kernels –Highpass Filters . . . . . . . . . . . . . 66
5.3 Nonlinear Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5.3.1 Median Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5.3.2 Example of Median Filter of Uniform Distribution . . . . . . . 75
5.4 Median Filter and Gaussian Noise . . . . . . . . . . . . . . . . . . . . 77
5.5 Comparison of Histograms after Mean and Median Filter . . . . . . . 79
5.6 E¤ect of Window “Shape” . . . . . . . . . . . . . . . . . . . . . . . . 79
5.6.1 Other Statistical Filters (Mode, Variance, Maximum, Minimum) 81
5.6.2 Examples of Nonlinear Filters . . . . . . . . . . . . . . . . . . 81
5.6.3 Nonlinear Filters on Images with Additive Gaussian Noise . . 82

5.6.4 Nonlinear Filters on Noise-Free Gray-Level Image . . . . . . . 82

5.7 Adaptive Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

5.8 Convolution Revisited –Bandpass Filters . . . . . . . . . . . . . . . . 82
5.9 Bandpass Filters for Images . . . . . . . . . . . . . . . . . . . . . . . 87
5.10 Implementation of Filtering . . . . . . . . . . . . . . . . . . . . . . . 87
5.11 Neighborhood Operations on Multiple Images . . . . . . . . . . . . . 88
5.11.1 Image Sequence Processing . . . . . . . . . . . . . . . . . . . . 88
5.11.2 Spectral + Spatial Neighborhood Operations . . . . . . . . . . 88
5.11.3 Image Division to Enhance Low-Contrast Image Structure . . 89

6 Shape-Based Operations 91
6.1 Pixel Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
6.2 Image Labeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.2.1 Example: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.3 Border Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
6.4 Cascaded Morphological Operations –“Opening”and “Closing” . . . 96
6.5 Applications of Morphological Operations . . . . . . . . . . . . . . . 97
6.5.1 Noise Removal . . . . . . . . . . . . . . . . . . . . . . . . . . 97
CONTENTS vii

6.5.2 Medial Axis Transform . . . . . . . . . . . . . . . . . . . . . . 98

6.6 Binary Morphological Operations . . . . . . . . . . . . . . . . . . . . 98
6.6.1 Horizontal Structuring Element . . . . . . . . . . . . . . . . . 98
6.6.2 Vertical Structuring Element . . . . . . . . . . . . . . . . . . . 99
6.6.3 “Complex”Structuring Element . . . . . . . . . . . . . . . . . 99
6.6.4 Binary Morphological Operations . . . . . . . . . . . . . . . . 99

7 Geometric Operations 101

7.1 Least-Squares Solution for Warping . . . . . . . . . . . . . . . . . . . 103
7.2 Common Geometrical Operations . . . . . . . . . . . . . . . . . . . . 106
7.3 Pixel Transfers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
7.4 Pixel Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

8 Global Operations 109

8.1 Relationship to Neighborhood Operations . . . . . . . . . . . . . . . 109
8.2 Discrete Fourier Transform (DFT) . . . . . . . . . . . . . . . . . . . . 110
8.3 Fast Fourier Transform (FFT) . . . . . . . . . . . . . . . . . . . . . . 111
8.4 Fourier Transforms of Images . . . . . . . . . . . . . . . . . . . . . . 114
8.5 Image Restoration via Fourier Transforms . . . . . . . . . . . . . . . 115
8.5.1 Examples of Inverse Filters in 1-D . . . . . . . . . . . . . . . . 117
8.5.2 Spectrum and Impulse Response of Inverse Filter . . . . . . . 118
8.5.3 Inverse Filter for SIN C-Function Blur . . . . . . . . . . . . . 118
8.6 Other Global Operations . . . . . . . . . . . . . . . . . . . . . . . . . 119
8.7 Discrete Cosine Transform (DCT) . . . . . . . . . . . . . . . . . . . . 119
8.7.1 Steps in Forward DCT . . . . . . . . . . . . . . . . . . . . . . 122
8.7.2 Steps in Inverse DCT . . . . . . . . . . . . . . . . . . . . . . . 123
8.8 Walsh-Hadamard Transform . . . . . . . . . . . . . . . . . . . . . . . 123

9 129
 Preface

References
Center for Image Processing in Education: lots of links to software and images
http://www.evisual.org/homepage.html
ImageJ software for image processing and analysis in Java, evolution of NIHImage
http://rsb.info.nih.gov/ij/
Image 2000 (from NASA)
http://www.ccpo.odu.edu/SEES/ozone/oz_i2k_soft.htm
Scion Image Processing Software (for PC and MAC-OS)
http://www.scioncorp.com/frames/fr_scion_products.htm
Hypercube Image Analysis Software (for PC and MAC-OS)
http://www.tec.army.mil/Hypercube/
GIMP Image Processing Software (Gnu-IMP). for PC, MacOS, Linux
http://www.gimp.org/
Irfanview (free image processing viewer with some processing capability)
http://www.irfanview.com/

Gregory A. Baxes, Digital Image Processing, Principles and Applications,

John Wiley & Sons, New York, 1994.
Ronald N. Bracewell, Two-Dimensional Imaging, Prentice Hall, Englewood
Cli¤s, 1995.
Ronald N. Bracewell, The Fourier Transform and Its Applications (Second
Edition, Revised), McGraw-Hill, 1986.
Ronald N. Bracewell, The Hartley Transform, Oxford University Press, New
York, 1986.
R.N. Bracewell, “The Fourier Transform”, Scienti…c American, June 1989,
pp.86-95.
Kenneth R. Castleman, Digital Image Processing, Prentice Hall, Englewood
Cli¤s, 1996.
E.O.Brigham, The Fast Fourier Transform and its Applications, Prentice
Hall, Englewood Cli¤s, 1988.
Michael P. Ekstrom, (Ed.), Digital Image Processing Techniques, Academic
Press, New York, 1984.
B.R. Frieden, Probability, Statistical Optics, and Data Testing, Third Edi-
tion, Springer-Verlag, Berlin, 2002.
Jack D. Gaskill, Linear Systems, Fourier Transforms, and Optics, John
Wiley & Sons, New York, 1978.
Rafael C. Gonzalez and Richard E. Woods, Digital Image Processing, Second
Edition, Prentice Hall, Upper Saddle River, 2002.
Jae S. Lim, Two-Dimensional Signal and Image Processing, Prentice Hall,
Englewood Cli¤s, 1990.
Paul J. Nahin, An Imaginary Tale, Princeton University Press, Princeton NJ,
1998.

ix
x Preface

A. Nussbaum and R. Phillips, Contemporary Optics for Scientists and En-

gineers, Prentice-Hall, 1976.
Wayne Niblack, An Introduction to Digital Image Processing, Prentice
Hall, Englewood Cli¤s, 1986.
J. Anthony Parker, Image Reconstruction in Radiology, CRC Press, Boca
Raton FL, 1990.
William K. Pratt, Digital Image Processing, Second Edition, John Wiley &
Sons, New York, 1991.
Azriel Rosenfeld and Avinash C. Kak, Digital Picture Processing, Second
Edition, Academic Press, San Diego, 1982.
Craig Scott, Introduction to Optics and Optical Imaging, IEEE Press, New
York, 1998.
J.S.Walker, Fast Fourier Transforms 2nd Edition, CRC Press, New York, 1996.
Chapter 1

Basic Principles of Digital Image

Processing

During the last two decades or so, inexpensive and powerful digital computers have
become widely available and have been applied to a multitude of tasks. By hitching
computers to new imaging detectors and displays, very capable systems for creating,
analyzing, and manipulating imagery have been constructed and are being applied
in many arenas. For example, they now are used to reconstruct X-ray and magnetic
resonance images (MRI) in medicine, to analyze multispectral aerial and satellite
images for environmental and military uses, to read Universal Product Codes that
specify products and prices in retail stores, to name just a few.

Since I …rst taught a predecessor of this course in 1987, the capabilities of inex-
pensive imaging tools (cameras, printers, computers) have exploded (no surprise to
you, I’m sure). This has produced a wide and ever-expanding range of applications
that we could not even envision in those days. To give an idea of the change over the
last two decades, consider the set of data shown below, which is copied directly from
the …rst edition of Digital Image Processing by Gonzalez and Woods:

1
2CHAPTER 1 BASIC PRINCIPLES OF DIGITAL IMAGE PROCESSING

Coded 64 64 5-bit image (32 gray values)

These data represent a 64 64 5-bit image (25 = 32 gray values). This data set was
entered by hand (with only 4 mistakes) in 1988 by Ranjit Bhaskar, an imaging science
graduate student for use by students. The image was rendered using the so-called
“overstrike”routine on a line printer, where “dark”pixels were printed using several
overprinted characters and lighter pixels by sparse characters (e.g. “.”and “-”). The
subject of the image is shown on the next page:
 3

Data in previous photo rendered using “overstrike” printout with line printer – this
is how we used to do it, folks!

This course will investigate the basic principles of digital imaging systems and
introduce useful applications; many simple examples will be provided to illustrate the
concepts. First, a de…nition:

IMAGE: A reproduction or imitation of form of a person or thing.

The optical counterpart of an object produced by a lens, mirror, etc.
..................................Noah Webster

We normally think of an image in the sense of a picture, i.e., a planar represen-

tation of the brightness, , i.e., the amount of light re‡ected or transmitted by an
object.
An image is usually a function of two spatial variables, e.g., f [x; y], which rep-
resents the brightness f at the Cartesian location [x; y]. Obviously, it also may be
graphed in three dimensions, with brightness displayed on the z-axis.
4CHAPTER 1 BASIC PRINCIPLES OF DIGITAL IMAGE PROCESSING

Image Representation of
f [n; m]

Function of Two Spatial Coordinates f [x; y]

It is more and more common to deal with images that have more than two coor-
dinate dimensions, e.g.,

f [x; y; tn ] monochrome “movie”, discrete set of images over time

f [x; y; ] spectral image with continuous domain of wavelengths
f [x; y; n] multispectral image, discrete set of wavelengths
f [x; y; t] time-varying monochrome image over continuous time domain
f [x; y; tn ] time-varying monochrome image with discrete time samples (cinema)
f [x; y; z] 3-D monochrome image (e.g., optical hologram)
f [x; y; tn ; m] discrete samples in time and wavelength, e.g., color movie
f [x; y; z; t; ] reality
It is generally fesible to “cut”2-D slices from these multidimensional functions to cre-
ate images, but the images need not be “pictorial.” For example, consider the 2-D
slices “cut” from the 3-D function spatial-temporal function f [x; y; t]; the 2-D slice
f [x; y; t = t0 ] is pictorial but f [x; y = y0 ; t] is not. That said, the units of the axes
have no e¤ect on the computations; it is perfectly feasible for computers to process
and display f [x; y = y0 ; t] as to do the same for f [x; y; t0 ].

After converting image information into an array of integers, the image can be
manipulated, processed, and displayed by computer. Computer processing is used
for image enhancement, restoration, segmentation, description, recognition, coding,
reconstruction, transformation
1.1 DIGITAL PROCESSING 5

1.1 Digital Processing

The general digital image processing system may be divided into three components:
the input device (or digitizer), the digital processor, and the output device (image
display).

1. The digitizer converts a continuous-tone and spatially continuous brightness

distribution f [x; y] to an discrete array (the digital image) fq [n; m], where n; m,
and fq are integers.

2. The digital processor operates on the digital image fq [n; m] to generate a new
digital image gq [k; `], where k, `, and gq are integers. The output image may be
represented in a di¤erent coordinate system, hence the use of di¤erent indices
k and `.

3. The image display converts the digital output image gq [k; `] back into a continuous-
tone and spatially continuous image g [x; y] for viewing. It should be noted that
some systems may not require a display (e.g., in machine vision and arti…cial
intelligence applications); the output may be a piece of information. For ex-
ample, a digital imaging system that was designed to answer the question, Is
there evidence of a cancerous tumor in this x -ray image?, ideally would have
two possible outputs (YES or NO), , i.e., a single bit of information.

Note that the system includes most of the links in what we call the imaging chain.
We shall …rst consider the mathematical description of image digitizing and display
devices, and follow that by a long discussion of useful processing operations.

1.2 Digitization
Digitization is the conversion of a continuous-tone and spatially continuous brightness
distribution f [x; y] to an discrete array of integers fq [n; m] by two operations which
will be discussed in turn:
6CHAPTER 1 BASIC PRINCIPLES OF DIGITAL IMAGE PROCESSING

(A) SAMPLING –a function of continuous coordinates f [x; y] is evaluated on

a discrete matrix of samples indexed by [n; m]. You probably saw some discussion of
sampling in the course on linear and Fourier mathematics.
(B) QUANTIZATION –the continuously varying brightness f at each sample
is converted to a one of set of integers fq by some nonlinear thresholding process.
The digital image is a matrix of picture elements, or pixels if your ancestors are
computers. Video descendents (and imaging science undergraduates) often speak of
pels (often misspelled pelz ). Each matrix element is an integer which encodes the
brightness at that pixel. The integer value is called the gray value or digital count of
the pixel.
Computers store integers as BInary digiTS, or bits (0,1)

2 bits can represent: 004 = 0:, 014 = 1, 104 = 2:, 114 = 3:;a total of 22 = 4
numbers.

(where the symbol “4 ”denotes the binary analogue to the decimal point “.”and
thus may be called the “binary point,”which separates the ordered bits with positive
and negative powers of 2).

m BITS can represent 2m numbers

=) 8 BITS = 1 BYTE =) 256 decimal numbers, [0; 255]
=) 12 BITS = 4096 decimal numbers, [0; 4095]
=) 16 BITS = 216 = 65536 decimal numbers, [0; 65535]

Digitized images contain …nite numbers of data “bits”and it probably is apparent

that the process of quantization discards some of the content of the image, i.e., the
quantized image di¤ers from the unquantized image, so errors have been created.
Beyond that, we can consider the ”amount”of “information”in the quantized image,
which is de…ned as the number of bits required to store the image. The number
of bits of “information” usually is smaller than the the number of bits of “data”
(which is merely the product of the number of image pixels and the number of bits
per pixel). The subject of information content is very important in imaging and will
be considered in the section on image compression. We will discuss digitizing and
reconstruction errors after describing the image display process.
Chapter 2

Review of Sampling

The process of “sampling” derives a discrete set of data points at (usually) uniform
spacing. In its simplest form, sampling is expressed mathematically as multiplication
of the original image by a function that “measures”the image brightness at discrete
locations of in…nitesimal width/area/volume in the 1-D/2-D/3-D cases:

fs [n x] = f [x] s [x; n x]
where:
f [x] = brightness distribution of input image
s [x; n x] = sampling function
fs [n x] = sampled input image de…ned at coordinates n x

The ideal sampling function for functions of continuous variables is generated from
the so-called “Dirac delta function” [x], which is de…ned by many authors, including
Gaskill. The ideal sampling function is the sum of uniformly spaced “discrete”Dirac
delta functions, which Gaskill calls the COMB while Bracewell calls it the SHAH :

X
+1
COM B [x] [x n]
n= 1
X
+1
1 h x i
s [x; n x] [x n x] COM B
n= 1
x x

7
8 CHAPTER 2 REVIEW OF SAMPLING

The COMB function de…ned by Gaskill (called the SHAH function by Bracewell).

For the (somewhat less rigorous) purpose we have here, we may consider the
sampling function to just “grab”the value of the continuous input function f [x; y] at
the speci…c locations separated by the uniform intervals x and y, where x = y:

fs [n; m] = f [n x; m y]

In other words, we are sweeping some unimportant and possibly confusing mathe-
matical details under the rug.

2.0.1 Ideal Sampling of 1-D function

Multiplication of the input f [x] by a COMB function merely evaluates f [x] on the
uniform grid of points located at n x, where n is an integer. Because it measures
the value of the input at an in…nitesmal point, this is a mathematical idealization
that cannot be implemented in practice. Even so, the discussion of ideal sampling
usefully introduces some essential concepts.
Consider ideal sampling of a sinusoidal input function with spatial period X0 that
is ideally sampled at intervals separated by x:

1 2 x
f [x] = 1 + cos + 0
2 X0
The amplitude of the function at the sample indexed by n is:

1 n x
fs [n x] = 1 + cos 2 + 0
2 X0
1 x
= 1 + cos 2 n + 0
2 X0

The dimensionless parameter Xx0 in the second expression is the ratio of the sampling
interval to the spatial period (wavelength) of the sinusoid and is a measurement of
 9

the …delity of the sampled image. For illustration, examples of sampled functions
obtained for several values of Xx0 are:

x 1 1 h ni
Case I: X0 = 12 x =) = ; 0 = 0 =) fs [n] = 1 + cos
X0 12 2 6
x 1 1 1
Case II: X0 = 2 x =) = ; 0 = 0 =) fs [n] = (1 + cos [ n]) = [1 + ( 1)n ]
X0 2 2 2
x 1 1 1
Case III: X0 = 2 x =) = ; 0= =) fs [n] = (1 + sin [ n]) =
X0 2 2 2 2
4 x 3
Case IV: X0 = x =) = ; 0=0
3 X0 4
1 2 n 1 h ni
=) fs [n] = 1 + cos = 1 + cos 3
2 4=3 2 2
4 x 5
Case V: X0 = x =) = ; 0=0
5 X0 4
1 2 n 1 h ni
=) fs [n] = 1 + cos = 1 + cos 8
2 5=4 2 5
10 CHAPTER 2 REVIEW OF SAMPLING

x
Illustration of samples of the biased sinusoids with the di¤erent values of X0
listed
in the table. The last three cases illustrate “aliasing.”
The output evaluated for Xx0 = 12 depends on the phase of the sinusoid; if sampled
at the extrema, then the sampled signal has the same dynamic range as f [x] (i.e.,
it is fully modulated), show no modulation, or any intermediate value. The interval
x = X20 de…nes the Nyquist sampling limit. If Xx0 > 12 sample per period, then the
same set of samples could have been obt5ained from a sinusoid with a longer period
and a di¤erent sampling interval x. For example, if Xx0 = 34 , then the reconstructed
function appears as though obtained from a sinudoid with period X00 = 3X0 if sampled
with Xx0 = 14 . In other words, the data set of samples is ambiguous; the same samples
0
2.1 ALIASING –WHITTAKER-SHANNON SAMPLING THEOREM 11

could be obtained from more than one input, and thus we cannot distinguish among
the possible inputs based only on knowledge of the samples.

2.1 Aliasing –Whittaker-Shannon Sampling The-

orem
As just demonstrated, the sample values obtained from a sinusoid which has been
sampled fewer than two times per period will be identical to those obtained from a
sinusoid with a longer period. This ambiguity about which original function produced
the set of samples is called aliasing in sampling, but similar e¤ects show up whenever
periodic functions are multiplied or added. In other disciplines, these go by di¤erent
names such as beats, Moiré fringes, and heterodyning. To illustrate, consider the
product of two sinusoidal functions with the di¤erent periods X1 and X2 (and thus
spatial frequencies 1 = X11 ; 2 = X12 ), which may be written as the sum of two
sinusoids with di¤erent spatial frequencies:
1 1
cos [2 1 x] cos [2 2 x] = cos [2 ( 1 + 2 )x] + cos [2 ( 1 2 )x]
2 2
(note that the converse is also true; the sum of two sinusoids with the same amplitude
and di¤erent frequencies may be written as the product of two sinusoids). The second
term in the expression for the product oscillates slowly and is the analog of the aliased
signal.
Though the proof is beyond our mathematical scope at this time, we can state
that a sinusoidal signal that has been sampled without aliasing may be reconstructed
without error from the (in…nite set of) its ideal samples. This will be demonstrated
in the section on image displays. Also without proof, we make the following claim:

Any 1-D or 2-D function may be expressed as a unique sum of 1-D or 2-D
sinusoidal components
with (generally) di¤erent amplitudes, frequencies, and phases.

This is the principle of Fourier analysis, which determines the set of sinusoidal
components from the function. The set of amplitudes and phases of the sinusoidal
components expressed as a function of frequency are the Fourier components of the
function.
If the sinusoidal representation of f [x] has a component with a maximum spatial
frequency max , and if we sample f [x] so that this component is sampled without alias-
ing, then all sinusoidal components of f [x] will be adequately sampled and f [x]can
be perfectly reconstructed from its samples. Such a function is band-limited and
max is the cuto¤ frequency of f [x]. The corresponding minimum spatial period is
1
Xmin = max . Thus the sampling interval x can be found from:

x 1 Xmin 1
< =) x< =) x<
Xmin 2 2 2 max
12 CHAPTER 2 REVIEW OF SAMPLING

This is the Whittaker-Shannon sampling theorem. The limiting value of the sam-
1
pling interval x = 2 max de…nes the Nyquist sampling limit. Sampling more or less
frequently than the Nyquist limit is oversampling or undersampling, respectively.
1
x> =) undersampling
2 max
1
x< =) oversampling
2 max

The Whittaker-Shannon Sampling Theorem is valid for all types of sampled sig-
nals. An increasingly familiar example is digital recording of audio signals (e.g., for
compact discs or digital audio tape). The sampling interval is determined by the
maximum audible frequency of the human ear, which is generally accepted to be
approximately 20kHz. The sampling frequency of digital audio recorders is 44,000
samples 1
second
which translates to a sampling interval of 44;000 s
= 22:7 s. At this sampling
rate, sounds with periods greater than 2 22:7 s = 45:4 s (or frequencies less than
(45:4 s) 1 = 22 kHz) can theoretically be reconstructed perfectly, assuming that
f [t] is sampled perfectly (i.e., at a point). Note that if the input signal frequency is
greater than the Nyquist frequency of 22 kHz, the signal will be aliased and will ap-
pear as a lower-frequency signal in the audible range. Thus the reconstructed signal
will be wrong. This is prevented by ensuring that no signals with frequencies above
the Nyquist limit is allowed to reach the sampler; higher frequencies are …ltered out
before sampling.

2.2 Realistic Sampling –Averaging by the Detec-

tor

In fact, it is not possible to “grab”the amplited of a signal at speci…c locations with

in…nitesimal “width” (in…nitesimal “support”). The measurement of a …nite signal
over the in…nitesimally small area in the real world would produce an in…nitesimal
result. Rather a real system performs “realistic sampling,”where the continuous input
is measured over …nite areas located at uniformly spaced samples by using a detector
with …nite spatial (or temporal) size. The measured signal is a spatial integral of
the input signal over the detector area, which blurs the image. For example, in the
common case where we assume that each sensor element has the same response over
its full area, we can calculate the sample value by integrating the signal over the
detector size. In the …gure, the sensor elements are separated by x and each has
width d0 :
2.2 REALISTIC SAMPLING –AVERAGING BY THE DETECTOR 13

For a biased sinusoidal signal of the form:

1 2 x
f [x] = 1 + cos + 0
2 X0

we de…ne its modulation as:

fmax fmin
m= ;0 m 1 if fmin 0
fmax + fmin

Note that modulation is only de…ned for nonnegative (i.e., biased) sinusoids. The
analogous quantity for a nonnegative square wave is called contrast.

The biased sinusoidal signal is averaged over the detector area, e.g., the sampled
value at n = 0 is:

Z +
d0
1 2
fs [n = 0] = f [x] dx
d0 d0
2
8
> d0
>
> 1 if jxj <
Z +1
>
< 2
1 x x 1 d0 d0
= f [x] RECT dx where: RECT if jxj = =) x =
d0 1 d0 d0 >
>
2
2 2
>
> d0
: 0 if jxj >
2
14 CHAPTER 2 REVIEW OF SAMPLING

For f [x] as de…ned above, the set of samples is derived by integrating f [x] over the
area of width d centered at coordinates that are integer multiples of x:
Z n x+
d0
1 2 1 2 x
1 + cos + 0 dx
d0 n x
d0
2
2 X0
Z d0 Z n x+ d0 !
n x+
1 2 2 2 x
= dx + cos + 0 dx
2d0 n x
d0
2
d
n x 20 X0
h i x=n x+
d0
2
2 x
1 d0 d0 1 sin X0
+ 0
= n x+ n x + 2
2d0 2 2 2d0 X0 d0
x=n x
h i h i 2

x d0 x d0
1 1 sin 2 n X0
+ X0
+ 0 sin 2 n X0 X0
+ 0
= +
2 2d0 2
X0

By de…ning
x
=2 n + 0
X0
d0
=
X0
and applying the trigonometric identities:

sin [ ] = sin cos cos sin

=) sin [ + ] sin [ ] = 2 cos sin ;

we …nd an expression for the integral over the detector area:

0 h i1
d0
1 1 @ x sin X0
fs [n] = + 2 cos 2 n + 0 A
2
2 2d0 X0 X0
0 h i1
d0
1 1 @ x sin X0
= + 2 cos 2 n + 0 d0 A
2 2d0 X0 2 Xd00

The last multiplicative term may be simpli…ed by applying the de…nition of the special
function
sin [ ]
SIN C [ ]
1 d0 x
=) fs [n] = 1 + SIN C cos 2 n + 0
2 X0 X0

which varies with period X0 and detector width d0 .

2.2 REALISTIC SAMPLING –AVERAGING BY THE DETECTOR 15

sin [ x]
SIN C [x] ; note that the value at the origin is unity, which may be
x
demonstrated via l’Hôpital’s rule.

Note that for constant functions, the period X0 = 1 and the resulting value of
the SIN C function is:
d0
lim SIN C =1
X0 !1 X0
which means that uniform weighted averaging has no e¤ect on any constant input
function. The samples of cosine of period X0 obtained with sampling interval x in
the two cases are:
1 d0 x
Realistic:fs [n] = 1 + SIN C cos 2 n + 0
2 X0 X0
1 x
Ideal: fs [n] = 1 + cos 2 n + 0
2 X0

where d0 is the width of the detector. The amplitude of the realistic case is multiplied
d0
by a factor of SIN C , which is less than unity everywhere except at the origin,
X0
, i.e., where d0 = 0 or X0 = 1. As the detector size increases relative to the
d0 d0
spatial period of the cosine ( i.e., as increases) , then SIN C ! 0 and the
X0 X0
modulation of the sinusoid decreases.

The modulation of the image of a sine-wave of period X0 , or spatial frequency

1 d0
= , is reduced by a factor SIN C = SIN C [d0 0 ].
X0 X0
16 CHAPTER 2 REVIEW OF SAMPLING

Example of Reduced Modulation due to Pre…ltering

The input function f [x] has a period of 128 units with two periods plotted. It is the
sum of six sinusoidal components plus a constant:

1 1 X ( 1)n
6 1
(2n 1) x
f [x] = + sin 2 :
2 2 n=1 n 256

The periods of the component sinusoids are:

128 1 cycles cycles
X1 = units =) 1 = ' 0:0078
1 128 unit unit
128 3 cycles cycles
X2 = units ' 42:7 units =) 2 = ' 0:023
3 128 unit unit
128 5 cycles cycles
X3 = units = 25:6 units =) 3 = ' 0:039
5 128 unit unit
128 7 cycles cycles
X4 = units ' 18:3 units =) 4 = ' 0:055
7 128 unit unit
128 9 cycles cycles
X5 = units ' 14:2 units =) 4 = ' 0:070
9 128 unit unit
128 11 cycles cycles
X6 = units ' 11:7 units =) 4 = ' 0:086
11 128 unit unit
The constant bias of 0.5 ensures that the function is positive. The …rst sinusoidal
component (X01 = 128 units) is the fundamental and carries most of the modulation
of the image; the other components (the higher harmonics) have less amplitude. The
spatial frequency of each component is much less than the Nyquist limit of 0.5.
2.2 REALISTIC SAMPLING –AVERAGING BY THE DETECTOR 17

Illustration of the reduction in modulation due to “pre…ltering”: (a) input function

f [n]; (b) result of pre…ltering with uniform averagers of width d = 0, d = X160 , and
d = X80 ; (c) magni…ed view of (b), showing the change in the signal; (d) result
o¢ ltering with uniform averagers of width d = X20 , d = X0 , and d = 0:75
X0
, showing
the “contrast reversal”in the last case.
18 CHAPTER 2 REVIEW OF SAMPLING

d0 1
SIN C = SIN C [d0 1 ] = SIN C 8 ' 0:994
X01 128
d0 3
SIN C = SIN C [d0 2 ] = SIN C 8 ' 0:943
X02 128
d0 5
SIN C = SIN C [d0 3 ] = SIN C 8 ' 0:847
X03 128
d0 7
SIN C = SIN C [d0 4 ] = SIN C 8 ' 0:714
X04 128
d0 9
SIN C = SIN C [d0 5 ] = SIN C 8 ' 0:555
X05 128
d0 11
SIN C = SIN C [d0 6 ] = SIN C 8 ' 0:385
X06 128

Note that the modulation of sinusoidal components with shorter periods (higher
frequencies) are diminished more severely by the averaging. A set of pre…ltered images
for several di¤erent averaging widths is shown on a following page. If the detector
width is 32 units, the resulting modulations are:

1
SIN C [d0 1 ] = SIN C 32 ' 0:900
128
3
SIN C [d0 2 ] = SIN C 32 ' 0:300
128
SIN C [d0 3] ' 0:180
SIN C [d0 4] ' 0:129
SIN C [d0 5] ' 0:100
SIN C [d0 6 ] ' +0:082

Note that the components with periods X04 and X05 have negative modulation,
, i.e., fmax < fmin . The contrast of those components is reversed. As shown, the
sampled image looks like a sawtooth with a period of 128 units.
If the detector size is 128, each component is averaged over an integral number of
periods and the result is just the constant bias; the modulation of the output is zero:

128
SIN C [d0 1 ] = SIN C = SIN C [1] = 0
128
7
SIN C [d0 2 ] = SIN C 128 = SIN C [3] = 0
42

For a detector width of 170 units, the modulations are:

2.2 REALISTIC SAMPLING –AVERAGING BY THE DETECTOR 19

1
SIN C [d0 1 ] = SIN C 170 ' 0:206
128
3
SIN C [d0 2 ] = SIN C 170 ' 0:004
128
5
SIN C [d0 3 ] = SIN C 170 ' +0:043
128
7
SIN C [d0 4 ] = SIN C 170 ' 0:028
128
SIN C [d0 5] ' 0:004
SIN C [d0 6 ] ' +0:021

Because the …rst (largest amplitude) sinusoidal component has negative modula-
tion, so does the resulting image. The overall image contrast is reversed; darker areas
of the input become brighter in the image.
Chapter 3

Review of Quantization

3.1 Quantization: A/D Conversion

The process of quantization converts the continuously valued irradiance (“lightness”
or “brightness”) measured at a sample (or a derived signal, such as the voltage mea-
sured from that signal) to one of a discrete set of gray levels (sometimes called dig-
ital counts though its acronym of “DC” may be confused with that for “direct cur-
rent”). For example, the measured signal at the sample located at [x0 ; y0 ] might be
W
f [x0 ; y0 ] = 1:234567890 mm2
. This number is converted to some integer value, per-
haps fq = 42. The range of allowed integers is determined by various factors in the
quantizer. For example, we might have 0 fq (fq )max where the maximum value
(fq )max is determined by the number of bits in the quantizer: (fq )max = 255 = 28 1
for an 8-bit quantizer and (fq )max = 212 1 = 4095 for the (increasingly common)
12-bit quantizer in scienti…c digital cameras.

3.1.1 Tone Transfer Function

1. Quantization is signi…cantly a¤ected by detector parameters, including its dy-
namic range and linearity. The dynamic range of a detector image may be
de…ned as the range of brightness (irradiance) over which a change in the in-
put signal produces a detectable change in the measured output of the sensor.
Note that the input and output quantities need not be identical; for example,
in old-fashioned emulsion photography, the input might be measured as power
W
per unit area (e.g., mm 2 ) and the resulting output in the dimensionless optical

density D where 0 D 1. The e¤ect of the detector on the measurement

may be described by a transfer characteristic or tone-transfer function (TTF,
also called the tone-transfer curve TTC), which is merely a graph of the out-
put value as a function of the various inputs to the sensor. The shape of the
transfer characteristic may be used as a …gure of merit for the measurement
process. A detector is linear if the TTC is a straight line, i.e., if an incremen-
tal change in input from any level produces a …xed incremental change in the
output. Of course, all real detectors have a limited dynamic range, i.e., they
will not respond at all to light intensity below some minimum value and their

21
22 CHAPTER 3 REVIEW OF QUANTIZATION

Figure 3.1: Possible tone-transfer function of quantizer applied to photographic emul-

sion, showing the “toe” on the left, “linear region” in the center, and “shoulder” on
the right.

response will not change for intensities above some maximum. All realistic de-
tectors are therefore nonlinear, but there may be some regions over which they
are more-or-less linear, with nonlinear regions at either end. A common such
example is photographic …lm; the TTC is the Hurter-Dri¢ eld (H&D) curve,
which is a graph of the recorded optical density of the emulsion as a function
W
of the logarithm of the input irradiance (which might be measured in mm 2 ).

Another very important example in digital imaging is the video camera, whose
TTC maps input light intensity to output voltage. The transfer characteristic
of a video camera is approximately a power law:

Vout = c1 Bin + V0

where V0 is the threshold voltage for a dark input and (gamma) is the exponent
of the power law. The value of depends on the speci…c detector but typical
values are ' 1:7 for a vidicon camera and ' 1 for an image orthicon. The
value of gamma for a digital sensor (CCD) is inherently very close to unity, but
are often modi…ed in software to approximate photographic emulsions.

The resolution, or step size b0 , of the quantizer is the di¤erence in the measured
signal at the center of adjacent gray levels. It makes little sense to quantize the
sampled signal with a resolution b0 that is smaller than the signal uncertainty due to
noise in the detector system. Thus the e¤ective number of levels is often less than
the maximum available as determined by the number of bits.
3.1 QUANTIZATION: A/D CONVERSION 23

Conversion from a continuous range to discrete levels requires a thresholding op-

eration (e.g.,truncation or rounding). Some range of input brightnesses will map to
W
a single output level, e.g., all measured irradiances between 0.76 and 0.77 mm 2 might

map to gray level 59. Threshold conversion is a nonlinear operation, i.e., the thresh-
old of a sum of two inputs is not necessarily the sum of the thresholded outputs. The
concept of linear operators will be discussed extensively later, but we should say at
this point that the nonlinearity due to quantization makes it inappropriate to analyze
the complete digital imaging system (digitizer, processor, and display) by common
linear methods. This problem is usually ignored, as is appropriate for large numbers
of quantized levels that are closely spaced so that the digitized image appears con-
tinuous. Because the brightness resolution of the eye-brain is limited, quantizing to
only 50 levels is satisfactory for many images; in other words, 6bits of data is often
su¢ cient for images to be viewed by humans.
Quantization is performed by some kind of digital comparator, which is an inher-
ently nonlinear electronic device that accepts an analog input and produces one of
two possible output values that depend on whether the analog input is smaller or
larger than some reference signal; if smaller, the output is a value that may be nor-
malized to “0”and if ) or if larger, the output is a “1”. For example, if the reference
of the comparator is set at 0:5 V and if the input is finput = 0:3 V, then the output
is Q f0:3 Vg = 0. If the input voltage is doubled finput = 0:6 V, then the output is
Q f0:6 Vg = 1. Note that this behavior clearly does not satisfy the requirement for a
linear operator L, where the output is doubled if the input is doubled:

if L ff g = g, then L f2f g = 2g for a linear operator L

The more general expression for a linear operator is::

( N )
X X
N
if L ffn g = gn , then L n fn = n gn for a linear operator L
n=1 n=1

where f ng is a set of numerical (generally complex-valued) weights.

The simplest quantizer converts an analog input voltage to a 1-bit digital output
and can be constructed from an ideal di¤erential ampli…er, where the output voltage
Vout is proportional to the di¤erence of the input signal Vin and some reference Vref
that is provided by a calibrated source:

Vout = (Vin Vref )

If the weighting factor is su¢ ciently large to approximate 1, then the output
voltage will be +1 if Vin > Vref and 1 if Vin < Vref . In this simple example,
we would assign the digital value “0” to a negative output where Vin < Vref and
“1” to a positive output such that Vin > Vref . A quantizer with better resolution
may be constructed by cascading several such digital comparators with di¤erent (but
generally evenly spaced) reference voltages. A digital translator converts the set of
comparator signals to the binary code. The 1-bit comparator and a 2-bit analog-to-
24 CHAPTER 3 REVIEW OF QUANTIZATION

Figure 3.2: Comparator and 2-Bit analog-to-digital converter (ADC). The comparator
may be interpreted as an ampli…er with “in…nite” gain, so that its output is a “high”
voltage if Vin > Vref and a “low” voltage otherwise. The schematic of the ADC con-
sists of 4 comparators whose reference voltages are determined by the voltage divider
with the resistor ladder.

digital converter (ADC) are shown in the …gure:

In the most common scheme of uniform quantization, the step size b is …xed at
the value:
fmax fmin
b=
2m 1
where fmax and fmin are the extrema of the measured irradiances of the image samples
and m is the number of bits of the quantizer.
If the darkest and brightest samples of a continuous-tone image have measured
irradiances fmin and fmax respectively, and the image is to be quantized using m bits
(2m graylevels), then we may de…ne a set of uniformly spaced levels fq that span the
dynamic range via:

f [x; y] fmin f [x; y] fmin

fq [x; y] = Q =Q (2m 1)
b fmax fmin

where Q f g represents the nonlinear truncation or rounding operation, e.g., Q f3:657g =

3 if Q is truncation or 4 if Q is rounding. The form of Q determines the location of
the decision levels where the quantizer jumps from one level to the next. The image
irradiances are reconstructed by assigning all pixels with a particular gray level fq to
the same irradiance value E [x; y], which might be de…ned by “inverting” the quan-
tization relation. The reconstruction level is often placed between the decision levels
b
by adding a factor :
2
3.2 QUANTIZATION ERROR (“NOISE”) 25

Emax Emin b
E^ [x; y] = fq [x; y] + Emin +
2m 1 2
Usually (of course), E^ [x; y] 6= E [x; y] due to the quantization, i.e., there will be
quantization error. The goal of optimum quantization is to adjust the quantization
scheme to reconstruct the set of image irradiances which most closely approximates
the ensemble of original values. The criterion which de…nes the goodness of …t and the
statistics of the original irradiances will determine the parameters of the quantizer,
e.g., the set of thresholds between the levels.

The quantizer just described is memoryless, i.e., the quantization level for a pixel
is computed independently that for any other pixel. The schematic of a memoryless
quantizer is shown below. As will be discussed, a quantizer with memory may have
signi…cant advantages.

3.2 Quantization Error (“Noise”)

The di¤erence between the true input irradiance (or brightness) and the corresponding
irradiance corresponding to the computed digital level is the quantization error at that
pixel:
[n x; m y] f [n x; m y] fq [n x; m y] :
Note that the quantization error is bipolar in general, i.e., may take on positive
or negative values. It often is useful to describe the statistical properties of the
quantization error, which will be a function of both the type of quantizer and the
input image. However, if the di¤erence between quantization steps (i.e., the width
of a quantization level) is b, is constant, the quantization error for most images may
be approximated as a uniform distribution with mean value h [n]i = 0 and variance
b2
h( 1 [n])2 i = . The error distribution will be demonstrated for two 1-D 256-sample
12
images. The …rst is a section of a cosine sampled at 256 points and quantized to 64
levels separated by b = 1:
26 CHAPTER 3 REVIEW OF QUANTIZATION

n
Statistics of quantization noise: (a) f [n] = 63 cos 2 1024 where 0 n 255 (one
quarter of a cycle); (b) after quantization by rounding to nearest integer; (c)
quantization error " [n] = f [n] fq [n], showing that 21 " + 12 ; (d) histogram of
256 samples of quantization error, which shows that the error is approximately
uniformly distributed.

The computed mean of the error 1 [n] = f1 [n] Qff1 [n]g is h 1 [n]i = 5:1
1
10 4 and the variance is h 21 [n]i = 0:08 ' 12 . That the distribution of the errors is
approximately uniform is shown by the histogram.
The second image is comprised of 256 samples of Gaussian distributed random
noise in the interval [0,63]. The image is quantized to 64 levels.Again, the error 2 [n]
is uniformly distributed in the interval [ 0:5; +0:5] with mean 4:09 10 2 ' 0 and
1
variance 2 = h 22 [n]i >= 0:09 ' 12 .
3.2 QUANTIZATION ERROR (“NOISE”) 27

Statistics of quantization error for Gaussian distributed random noise: (a) f [n] with
= 27:7; = 11; (b) after quantization by rounding to nearest integer; (c)
quantization error " [n] = f [n] fq [n], showing that 12 " + 12 ; (d) histogram of
256 samples of quantization error, which shows that the error is approximately
uniformly distributed.

The total quantization error is the sum of the quantization error over all pixels in
the image:
XX
= [n x; m y] :
i j

An image with large bipolar error values may thus have a small total error. The
mean-squared error (the average of the squared error) is a better descriptor of the
…delity of the quantization:

2 1 XX 2
= [n x; m y] ;
N i j
28 CHAPTER 3 REVIEW OF QUANTIZATION

W 2
where N is the number pixels in the image. If the irradiance is measured in mm 2;
W 2
will have units of mm 2 , so it is common to evaluate the square root to compute
the root-mean-squared error (RMS), which has the same dimensions as the error (or
as the signal):
s
p
2 =
1 XX 2
RMS Error = [n x; m y]:
N i j

It should be obvious that the RMS error obtained for one image using di¤erent quan-
tizers will depend on the choice used, and that the RMS error from one quantizer will
di¤er for di¤erent images. It should also be obvious that it is desirable to minimize
the RMS error in an image. The brute-force method for minimizing quantization
error is to add more bits to the ADC, which increases the cost of the quantizer and
the memory required to store the image. There also is a limit to the e¤ectiveness of
such an approach, since the output from a quantizer with a very small step size will
be noisy.

3.2.1 Signal-to-Noise Ratio

The signal-to-noise power ratio of an analog signal is most rigorously de…ned as the
dimensionless ratio of the variances of the signal and noise:
2
f
SN R 2
n

2
where the variance of a signal is a measure of the spread of its amplitude about
the mean value:
Z +1 Z +
X0
2 1 2
2
f = [f [x] hf [x]i] dx ! [f [x] hf [x]i]2 dx
1 X0 X0
2

Thus a large SNR means that there is a larger variation of the signal amplitude
than of the noise amplitude. The de…nition of SNR as the ratio of variances may
have a large range of values –easily several orders of magnitude –and the numerical
values may become unwieldy. The range of SNR may be compressed by expressing it
on a logarithmic scale as a dimensionless quantity called a bel:
2 2
f f f
SN R = log10 2
= log10 = 2 log10 [bels]
n n n

This de…nition of SNR is even more commonly expressed in units of tenths of a bel,
or decibels, so that the integer value is more precise:
2
f f
SN R = 10 log10 2
= 20 log10 [decibels]
n n
3.2 QUANTIZATION ERROR (“NOISE”) 29

Under this de…nition, the SNR is 10 dB if the signal variance is ten times larger than
the noise variance and 20 dB if the standard deviation is ten times larger than that
of the noise.
The variances obviously depend on the independent statistics (speci…cally on the
histograms) of the signal and noise. The variances do not depend on the “arrange-
ment”of the gray levels (i.e., the numerical “order”or “pictorial”appearance of the
pixel gray values) in the image. Since the noise often is determined by the mea-
surement equipment, a single measurement of the noise variance often is used for
many signal amplitudes. However, the signal variance must be measured each time.
Consider a few examples of common 1-D signals.

3.2.2 Example: Variance of a Sinusoid

The variance of a sinusoid with amplitude A0 is easily computed by direct integration:

x
f [x] = A0 cos 2 =) hf [x]i = 0
X0
Z +
X0 Z +
X0
2
2 1 2
2 1 2 x
f = [f [x] hf [x]i] dx = A0 cos 2 dx
X0 X0
2
X0 X0
2
X0
Z X
+ 20
A20 1 x A20
= 1 + cos 4 dx = (X0 + 0)
X0 X0
2
2 X0 2X0

where the trigonometric identity

1
cos2 [ ] = (1 + cos [2 ])
2
has been used. The …nal result is:

2 A20
f = for sinusoid with amplitude A0
2

Note that the variance of the sinusoid does not depend on the period (i.e., on the
spatial frequency) or on the initial phase – it is a function of the histogram of the
values in a period and not of the “ordered” values. It also does not depend on any
“bias” (additive constant) in the signal. The standard deviation of the sinusoid is
just the square root of the variance:
A0
f = p for sinusoid with amplitude A0
2
30 CHAPTER 3 REVIEW OF QUANTIZATION

3.2.3 Example: Variance of a Square Wave:

The variance of a square wave also is easily evaluated by integration of the thresholded
sinusoid:
x
f [x] = A0 SGN cos 2 =) hf [x]i = 0
X0
Z X0 Z X0 Z 3X0
!
+ + +
1 2 1 4 4
2
f = [f [x] hf [x]i]2 dx = ( A0 )2 dx + (+A0 )2 dx
X0 X0
2
X0 X0
4
X
+ 40

1 X0 X0
= A20 + A20 = A20
X0 2 2
2
=) f = A20 for square wave with amplitude A0
=) f = A0 for square wave with amplitude A0

Note that:
( f )square wave, amplitude A0 >( f )sinusoid, amplitude A0

which makes intuitive sense, because the amplitude of the square wave is often more
“distant”from its mean than the sinusoid is.

3.2.4 Variance of “Noise” from a Gaussian Distribution

A set of amplitudes selected at random from a probability distribution with a Gaussian
“shape”is call Gaussian noise. The most common de…nition of the Gaussian distrib-
ution is: " #
1 (x )2
p [n] = p exp
2 2 2 2
where is the mean value of the distribution and 2 is the variance. This shows
that the Gaussian distribution is completely speci…ed by these two parameters. The
standard deviation is a measure of the “width”of the distribution and so in‡uences
the range of output amplitudes.

3.2.5 Approximations to SNR

Since the variance depends on the statistics of the signal, it is common (though less
rigorous) to approximate the variance by the square of the dynamic range (peak-to-
peak signal amplitude fmax fmin f ). In most cases, ( f )2 is larger (and often
much larger) than f2 . In the examples of the sinusoid and the square wave already
considered, the approximations are:

A20
Sinusoid with amplitude A0 =) 2
f = ; ( f )2 = (2A0 )2 = 4A20 = 8 2
f
2
Square wave with amplitude A0 =) 2
f = A20 ; ( f )2 = (2A0 )2 = 4A20 = 4 2
f
3.2 QUANTIZATION ERROR (“NOISE”) 31

Figure 3.3: One exemplar and the histogram of 8192 samples from the Gaussian
distribution
" with mean# value = 4 and standard deviation = 2: p [n] =
2
1 n 4
p exp
2 2

For Gaussian noise with variance 2 = 1 and mean , the dynamic range of the noise
technically is in…nite, but since few amplitudes exist that are outside of the interval
of four standard deviations measured from the mean. This means that the extrema
of the Gaussian distribution may be approximated by fmax = + 4 , fmin = 4 ,
2
leading to f = 8 . The estimate of the variance of the signal is then ( f ) = 64 f2 ,
which is (obviously) 64 times larger than the actual variance. Because this estimate
of the signal variance is too large, the estimates of the SNR thus obtained will be too
optimistic.
Often, the signal and noise of images are measured by photoelectric detectors as
di¤erences in electrical potential in volts; the signal dynamic range is Vf = Vmax Vmin ,
the average noise voltage is Vn , and the signal-to-noise ratio is:

Vf2 Vf
SN R = 10 log10 = 20 log10 [dB]
Vn2 V

As an aside, we mention that the signal amplitude (or level) of analog electrical signals
often is described in terms of dB measured relative to some …xed reference signal. If
the reference level is V = 1 V (regardless of the impedance), then the signal level is
measured in units of “dBV:”

level = 10 log10 Vf2 dBV = 20 log10 (Vf ) dBV

The level is measured relative to 1 mV (across an impedance of 75 ) is in units of

32 CHAPTER 3 REVIEW OF QUANTIZATION

“dBmV:”
!
2
Vf
level = 10 log10 dBV = 60 log10 (Vf ) dBmV
10 3 V

3.2.6 SNR of Quantization

We now …nally get to the goal of this section: to determine the signal-to-noise ratio
of quantization of a signal. Clearly the quantized signal exhibits an error from the
original analog signal, which leads to the quantity of “noise” and thus a measure of
SNR. . Though in a strict sense the input signal and the type of quantizer determine
the probability density function of the quantization error, it usually is appropriate to
assume that the error due to quantization is uniformly distributed, i.e., the probability
density function is a rectangle. In the case of an m-bit uniform quantizer (2m gray
levels) where the levels are spaced by intervals of width b0 over the full analog dynamic
range of the signal, the error due to quantization will be (approximately) uniformly
distributed over this interval b0 ; this will be demonstrated in the following examples.
If the nonlinearity of the quantizer is rounding, the mean value of the error is 0; if
truncation to the next lower integer, the mean value is b20 . Any book on probability
and statistics likely will demonstrate that the variance and standard deviation of
uniformly distributed noise over a range b0 are:

2 b20
n =
12
r
b20 b0
n = =p
12 12
This also may be derived quite easily by evaluating the terms in the expression for
the variance:
2
n = n
2
hni2
where the noise function is
" #
1 n b20
pn = RECT
b0 b0

b0
This means that the mean value hni = and the mean value of n2 is:
2

Z Z " #
2
+1
1 +1 2 n b20
n = n2 p [n] dn = n RECT dn
1 b0 1 b0
Z b0 n=b0
1 1 n3 1 b30 b2
= n2 dn = = = 0
b0 0 b0 3 n=0 b0 3 3
3.2 QUANTIZATION ERROR (“NOISE”) 33

So the variance is:

2
n = n2 hni2
2
b2 b0 1 1 b20
= 0 = b20 = QED
3 2 3 4 12
2 b20
n = for uniformly distributed noise over range b0
12

For an m-bit quantizer and a signal with with maximum and minimum amplitudes
fmax and fmin (and range f = fmax fmin ), the width b0 of a quantization level (the
step size) has the value:
fmax fmin f
b0 = = ( f) 2 m
2m 2m
If we can assume that the quantization noise is uniformly distributed, then the vari-
ance of the quantization noise is well de…ned:
2
b2 1 f 1
2
n = 0 = = ( f )2 12 22m
12 12 2m

The SNR is the ratio of the variance of the signal to that of the noise due to quanti-
zation:
2 2m
f 2 12 2
= f
2
n ( f )2
This may be expressed on a logarithmic scale to yield the (sort of) simple expression:

SN R = 10 log10 2
f 12 22m 10 log10 ( f )2
= 10 log10 2
f + 10 log10 [12] + 20 m log10 [2] 10 log10 ( f )2
= 10 log10 2
f + 10 1:079 + 20 m 0:301 10 log10 ( f )2
2
f
= 6:02 m + 10:8 + 10 log10 [dB]
( f )2

The third term obviously depends on the parameters of both the signal and the quan-
tizer. This equation demonstrates one result immediately – that the SNR of quan-
tization increases by ' 6 dB for every bit added to the quantizer (again, assuming
uniform distribution of the noise). If using the (poor) estimate that f2 = ( f )2 , then
the third term evaluates to zero and the approximate SNR is:

SN R for quantization to m bits = 6:02 m + 10:8 + 10 log10 [1]) = 6:02 m + 10:8 [dB]

The statistics (and thus the variance) may be approximated for many types of
signals (e.g., music, speech, realistic images) as resulting from a random process.
The histograms of these signals usually are peaked at or near the mean value and
the probability of a gray level decreases for values away from the mean; the signal
34 CHAPTER 3 REVIEW OF QUANTIZATION

approximately is the output of a gaussian random process with variance f2 . By

selecting the dynamic range of the quantizer f to be su¢ ciently larger than f , few
(if any) levels should be clipped by the quantizer. For a Gaussian random process,
virtually none of the values will be clipped if the the maximum and minimum levels
of the quantizer are four standard deviations from the mean level:
f
fmax f = f fmin = =4 f
2
In other words, we may choose the step size between levels of the quantizer to satisfy
the criterion:
2
f 1
f = 8 f =) 2 =
( f) 64
In this case, the SNR of the quantization process becomes:

1
SN R = 6:02 m + 10:8 + 10 log10
64
= 6:02 m + 10:8 + 10 ( 1:806)
= 6:02 m 7:26 [dB]

which is = 18 dB less than the (optimistic) estimate obtained by assuming that

2 2
f = ( f) .

This expression for the SNR of quantizing a Gaussian-distributed random signal

with measured variance f2 may be demonstrated by quantizing that signal to m bits
over the range fmin = 4 f to fmax = + 4 f , and computing the variance of the
2
quantization error n . The resulting SNR should satisfy the relation:
2
f
SN R = 10 log10 2
= (6:02 m 7:26) dB
n

This equation may be applied to demonstrate that the SNR of a noise-free analog
signal after quantizing to 8 bits is = 41 dB. The SNR of the same signal quantized
to 16 bits (common in CD players) is approximately 89 dB. The best SNR that can
be obtained from analog recording (such as on magnetic tape) is about 65 dB, which
is equivalent to that from a signal digitized to 12 bits per sample or 4096 levels.

The ‡ip side of this problem is to determine the e¤ective number of quantization
bits after digitizing a noisy analog signal. This problem was investigated by Shannon
in 1948. The analog signal is partly characterized by its bandwidth [Hz], which
is the analog analogue of the concept of digital data rate [bits per second]. The
bandwidth is the width of the region of support of the signal spectrum (its Fourier
transform).
3.3 QUANTIZATION TO FEW BITS 35

3.3 Quantization to Few Bits

If quantizing to a small number of bits, say 4 or fewer so that the number of gray
values is 16 or fewer, then the image appearance often su¤ers markedly. A special
case is quantization to a single bit for two gray values (black and white); this is called
halftoning from its original use to convert gray-scale images (photographs) to bitonal
images for printing in newspapers or books.

3.3.1 Improved Gray Scale (IGS) Quantization

The IGS quantization is a standard technique (considered in Gonzalez and Woods)
that has a rather glori…ed name for a fairly simple concept. The process is well
de…ned and reduces the e¤ects of false contouring by adding a random number of
size approximately equal to the quantization step size to the gray value of each pixel
BEFORE quantization. The recipe for the process is:

1. Set initial SUM to binary value (0000 0000)2

2. If most sign…cant four bits of current pixel evaluate to 1111, then set the new
sum to those four bits + 0000, otherwise set new sum with the four most
signi…cant bits plus the 4 least sign…cant bits of the old sum.

Example

Pixel Index Gray Binary Code Sum of 8-bit binary and 4-bit error 4-bit IGS Gray

n 1 N/A 0000 0000 0000 N/A

n 108 (0110 1100)2 0110 1100 + 0000 0000 = (0110) (1100) 01102 = 6:
n+1 139 (1000 1011)2 1000 1011 + 0000 (1100) = (1001) (0111) 10012 = 9:
n+2 135 (1000 0111)2 1000 0111 + 0000 (0111) = (1000) (0111) 10002 = 8:
n+3 244 (1111 0100)2 1111 0100 + 0000 (0111) = (1111) (1011) 11112 = 15:
n+4 200 (1100 1000)2 1100 1000 + 0000 (0000) = (1100) 1000 11002 = 12:

3.3.2 Quantizers with Memory –Error Di¤usion

Another way to reduce quantization error is to use a quantizer with memory, so that
the quantized value at a pixel is determined in part by the quantization error at
nearby pixels. A schematic diagram of the quantizer with memory is shown below:
36 CHAPTER 3 REVIEW OF QUANTIZATION

Flow chart for “quantizer with memory” = error-di¤usion quantization

A simple method for quantizing with memory that generally results in reduced
total error without a priori knowledge of the statistics of the input image and without
adding much additional complexity of computation was introduced by Floyd and
Steinberg (SID 36, 1976) as a means to simulate gray level images on binary image
displays and is known as error di¤usion. It is easily adapted to multilevel image
quantization. As indicated by the name, in error di¤usion the quantization error is
from one pixel is used to in the computation of the levels of succeeding pixels. In
its simplest form, all quantization error at one pixel is added to the gray level of the
next pixel before quantization.

1-D Error Di¤usion of 1-D Functions

In the 1-D case, the quantization level at sample location x is the gray level of the
sample minus the error [x 1] at the preceding pixel:

fq [x] = Q ff [x] [x 1]g

[x] = f [x] fq [x]
= f [x] Q ff [x] [x 1]g

Usually, the error is reset to zero at the beginning of each line. You can see that
this is simple to implement and the results are often very useful. Several years back,
John Knapp (CIS undergraduate and graduate student) used 1-D error di¤usion to
improve the reconstructions of computer-generated holograms for several di¤erent
applications and the results were often excellent (§23.4 in my book Fourier Methods
in Imaging).

2-D Error Di¤usion

In the 2-D case, it is possible (and often desirable) to divide and apply the resulting
quantization error weight among di¤erent as-yet unquantized pixels. The “standard”
Floyd-Steinberg algorithm quantizes the N N image in a raster from the upper
left corner (sequence of pixels: …rst row from left to right, then second row from left
to right, etc. to the bottom line of pixels). At each pixel, the quantization error is
7
evaluated and divided among four pixels yet to be quantized: 16 of this result is added
5 3
to the next pixel in line, 16 to the pixel below, 16 to the pixel below and “forward”,
3.3 QUANTIZATION TO FEW BITS 37

1
and 16
to the pixel below and “afterward.”A …gure is helpful:

7
16
3 5 1
16 16 16

The end result is an array of bitonal pixels (black and white) where the density of
white pixels is proportional to the gray value. The examples demonstrate the e¤ects
of binary quantization on gray-level images. The images of the ramp demonstrate
that why the binarizer with memory is often called pulse-density modulation. Note
that more …ne detail is preserved in binarized images from the quantizer with memory
(e.g.,the sky in Liberty and the shoulder in Lincoln). This is accomplished by possibly
enhancing the local binarization error.

2-D error-di¤used quantization for three gray-scale images: (a) linear ramp, after
quantizing at the midgray level, after Floyd-Steinberg error di¤usion at the midgray
level; (b) same sequence for "Lincoln"; (c) same sequency for "Liberty." The
error-di¤used images convey more information about the more raplidly varying
structure in the scenes.
38 CHAPTER 3 REVIEW OF QUANTIZATION

Error di¤usion has applications beyond quantization – for example, Eschbach and
Knox have used it to create algorithms for edge enhancement.
A discussion of the use of error di¤usion in ADC was given by Anastassiou (IEEE
Trans. Circuits and Systems, 36, 1175, 1989).