9/15/2009

Butterfly Nebula: Gas heated at 36000 F and traveling at 600 000 miles/hr (earth-moon
in 24 min!) from a dying star (used to be 5xsun) . Image was captured on 27 July 2009
by a Wide Field Camera (ultraviolet and visible light) onborad Hubble telescope. 2.1
Ref.: http://www.nasa.gov/mission_pages/hubble/multimedia/ero/index.html

Chapter 2: Digital Image Fundamentals

Other optical illusions

2.2
http://www.grand-illusions.com/square.htm

1
 9/15/2009

Chapter 2: Digital Image

Fundamentals

O tli
Outline
Elements of Visual Perception
Light and the Electromagnetic Spectrum
Image Sensing and Acquisition
Sampling and Quantization

2.3

Chapter 2: Digital Image Fundamentals

Structure of the Human Eye
The iris acts as a diaphragm to control the amount of light entering the eye. Front of eye is covered
by a transparent
surface called the

Light entering the cornea is focused to the retina surface by a

The retina is
Three membranes enclose the eye: composed of
Inside the
Cornea and sclera,
sclera Choroid
Choroid, Retina choroid is the two types of
receptors:
rods and
ciliary iris
body diaphragm cones.
Pupil size: 2-8mm Nerves connecting to the retina leave the
2.4
eyeball through the optic nerve bundle.
Eye color: melanin (pigment) in iris

2
 9/15/2009

Chapter 2: Digital Image Fundamentals

The distribution of rods and cones is radially symmetric wrt the fovea (central portion
of the retina), except at the blind spot which includes no receptors.

Cones are responsible for photopic (color or bright-light) vision; while rods are for
scotopic (dim-light) vision.

Fovea area in the retina is circular with 1.5 mm in diameter where most of the cones
are concentrated with 150 000 cones/mm2. This is easily achievable with medium 2.5
resolution CCD imaging chip of size 5mm x 5mm!

Structure of the Retina

Light
g receptors
p in the retina consist
of two types: rods and cones.

Rods are long slender receptors,

75~150 million, and
cones are shorter and thicker,
6~7 million.

2.6

3
 9/15/2009

Chapter 2: Digital Image Fundamentals

How’s an object seen at the back of the eye?

H
h

L l
The focal lenght (distance bet center of the lens and the retina)
varies
i from
f 17 mm to
t 14 mm (as
( the
th refractive
f ti power off the th lens
l
increases from its minimum to its maximum). Recall that H/L = h/l

Perception takes place by the relative excitation of light receptors,

which transform radiant energy into electrical impulses that are
ultimately decoded by the brain.
2.7

Chapter 2: Digital Image Fundamentals

Human eye can adapt to an

enormous range (in the order
of 1010) of light intensity
levels, from scotopic
threshold to the glare limit.
Subjective brightness (i.e. perceived intensity)
is a logarithmic function of the light intensity
incident on the eye.
y

In photopic vision alone, the range is about 106 (-2 to 4 in the log scale).
The transition from scotopic to photopic vision is gradual over the
range (0.001, 0.1) millilambert1 (-3 to -1 mL in the log scale).2
•1Johann H. Lambert 1777, German Physicist,
2.8
•2 see http://www.cns.nyu.edu/~msl/courses/2223/notes.2.pdf

4
 9/15/2009

Chapter 2: Digital Image Fundamentals

The visual system is not able to operate over such a huge range
simultaneously, instead,
simultaneously instead it changes its overall sensitivity.
sensitivity This
phenomena is called brightness adaptation.
For example, if the eye is adapted to brightness
level Ba, the short intersecting curve represents
the range of subjective brightness perceived by
the eye.

The range is rather restricted, i.e. below level Bb,

all stimuli are perceived as indistinguishable blacks.

The upper part of the curve (dashed line) is not

restricted, but when extended too far, it looses its
meaning as it raises the adaptation level higher than Ba.
2.9

Chapter 2: Digital Image Fundamentals

Experiment for brightness discrimination:

Look at a flat, uniformly illuminated large area, e.g. a large opaque

glass illuminated from behind by a light source with intensity I. Add
an increment of illumination ΔI, in the form of a short duration flash
as a circle in the middle. Vary ΔI and observe the result.

I+ΔI1
I+ΔI2
I
The results should move from ”no perceivable change” to ”perceived
change”. The fraction ΔIc/I for which ΔIc produces ”just perceivable
change” is called the Weber ratio. 2.10

5
 9/15/2009

Chapter 2: Digital Image Fundamentals

A small Weber ratio indicates ”good” brightness

where a small percentage change in illumination
is discriminable. On the other hand, a large Weber
ratio represents ”poor” brightness indicating that
a large percentage change in intensity is needed.

The curve shows that brightness discrimination

Rods at work
is poor (large Weber ratio) at low level of
illumination, and it improves significantly
(Weber ratio decreases) as background Cones at work
illumination increases.

The two branches illustrate the fact that at

low levels of illumination, vision is carried
out by the rods, whereas at high levels (showing
better discrimination), cones are at work. 2.11

Chapter 2: Digital Image Fundamentals

Perceived brightness is
NOT a simple function
of intensity.
Example 1: Mach bands
The reflected light intensity from each
strip is uniform over its width and differs
from its neighbors by a constant amount;
nevertheless, the virtual appearance is that
transitions at each bar appear brighter on
the left side and darker on the right side
(scalloped bands).

The Mach band* effect can be used to

estimate the impulse response of the
visual system.

*Mach 1906. 2.12

6
 9/15/2009

Chapter 2: Digital Image Fundamentals

Example 2: Simoultaneous Contrast

Each small square
q is actually
y the same intensity,
y, but because of different
intensities of the surrounding, the small squares do not appear equally bright.

Example 3: Metameric Pairs

Any two objects which appear equally bright, even though, their intensities are
different are called metameric pairs. 2.13

Chapter 2: Digital Image Fundamentals

Other optical illusions

2.14

http://www.grand-illusions.com/square.htm

7
 9/15/2009

Chapter 2: Digital Image Fundamentals

Other optical illusions

http://www.optillusions.com/
http://video.google.com/videoplay?docid=6330601890396636382
&q=nice+video

2.15

Chapter 2: Digital Image Fundamentals

Definition:
Light is an electromagnetic radiation which, which by
simulation, arouses a sensation on the visual receptors
making sight possible.

Sir Isaac Newton (1666) discovered that when a beam

of sunlight is passed through a glass prism, the
g g beam of light
emerging g is not white but consists
instead of a continuous spectrum of colors ranging
from violet to red. This is called the visible region of
the spectrum, see next figure.

2.16

8
 9/15/2009

Chapter 2: Digital Image Fundamentals

The Electromagnetic Spectrum
high energies

high frequencies

short wavelegths

2.17

Chapter 2: Digital Image Fundamentals

The electromagnetic spectrum can be expressed in terms

of wavelength (λ),
(λ) frequency (v),(v) or energy (E).
(E)
Recall that
λ = c/v
where c is the speed of light (2.998 x 108 m/s).
The energy of the various components is given by:
E = hv
where h is Planck’s constant (6.62606891 x 10-34 Joule-
seconds (or m2kg/s)). E is measured in electron-volt.
2.18

9
 9/15/2009

Chapter 2: Digital Image Fundamentals

Electromagnetic waves can be visualized as propagating sinusoidal

waves of varying wavelengths (λ) or as a stream of massless
particles, each traveling in a wavelike pattern and moving at the
speed of light. Each massless particle contains a certain amount (or
bundle) of energy. Each bundle of energy is called a photon.

λ is measured in meters (or km for radio waves), microns (visible)

or nanometers (for X-ray). 2.19

Image Sensing and Acquisition

The types of images we’re interested in are

generated by the combination of an
”illumination” source and the reflection or
absorption of energy from the ”scene” being
imaged. Exception: stained glass which transmits light
Rather than reflecting or absorbing it!

”Illumination”
”Ill i i ” includes
i l d visible
i ibl light,
li h radar,
d
infrared, X-ray, or ultrasound.
”Scene” may be a familiar 3D object,
underground, human internal organs.
2.20

10
 9/15/2009

How to transform illumination energy into digital images?

A single imaging sensor,

eg. a photodiode

voltage output is then digitized

to produce a digital image

A line sensor

An array sensor

2.21

Chapter 2: Digital Image Fundamentals

G
Generating
ti a 2-D
2 D iimage using
i a single
i l sensor.

This type of mechanical digitizers is called a microdensitometer

and is used in high-precision scanning (but slow). 2.22

11
 9/15/2009

Image Acquisition Using Sensor Strips

most
ost flat
at bed sca
scanner
e use linear
ea
sensor strips.

circular sensor strips are used, e.g. in medical and industrial imaging
to produce cross-sectional ”slice” images of 3-D objects.
2.23

Some Digital
Cameras
24 February 2009 – The world’s smallest
and lightest creative D-SLR with built-in image
stabilization has finally arrived! The new Olympus E-620
combines the technical sophistication required by pros with
easy-to-use functions desired by hobbyists. As a result,
ambitious photographers everywhere can now take
creativity to a whole new level. Outfitted with a custom 7-
point Twin autofocus system, the E-620 provides consistent
focus, as well as a generous 12.3 Megapixel High-Speed
Live MOS. Additionally, the Live View technology as well
as the 2.7” free-angle HyperCrystal III LCD make framing
every shot a cinch. In-camera Art Filters entice users to be
artistic by allowing them to apply stylish effects at the
touch of a button. Indeed, the EE-620
620 is everything that
makes Olympus Four Thirds Standard D-SLR cameras
great. The latest addition to the E-System range offers
creative and technological power – all rolled into one. The
new model will be available for purchase at the end of April
2009.

Ref.:
http://www.cameratown.com/news/news.cfm/hurl/id|7238
2.24

12
 9/15/2009

Some Digital Cameras

15.1 Megapixel APS-C CMOS sensor

6.3fps continuous shooting, max. burst 90 JPEGs with UDMA card
DIGIC 4 processor
ISO 100-3200, expandable to 12800
9-point wide area AF
3.0” Clear View VGA LCD with Live View mode & Face Detection Live AF
Magnesium alloy body, with environmental protection
EOS Integrated Cleaning System
HDMI connection for high quality viewing and playback on a High Definition TV
Full compatibility with Canon EF and EF-S lenses and EX-series Speedlites 2.25

Some Digital Cameras

Main Features

•7x Zoom-NIKKOR lens

S
•Sure G
Grip
i with
i h reassuring
i fit
fi to capture all
ll those
h precious
i moments
•12.0 effective megapixels for high-resolution images
•2.7-in. high-resolution LCD monitor
•High performance image sensor shift VR image stabilization
•Motion Detection for sharp, steady results
•High Sensitivity up to ISO 6400*
•Sport Continuous Mode for high-speed capture settings
•Scene Auto Selector provides quick, carefree picture-taking in a variety
of situations
•Smart Portrait System ? Face-priority AF, Smile Mode, Blink Proof and
In-Camera Red-Eye y Fix
•Quick Retouch for the best balance of contrast and saturation
•D-Lighting adds detail and optimizes exposure to rescue underexposed
images

Ref.:
http://imaging.nikon.com/products/imaging/lineup/digitalcame
ra/coolpix/s630/index.htm
2.26

13
 9/15/2009

Some Digital Cameras

I t
Integrated
t d Digital
Di it l Camera
C
VGA 640 x 480 resolution 5 mega-pixel, (2592 x 1944 pixels), Carl Zeiss
Three file sizes for images: –optical lens
P=Photo: max 60 kB MPEG-4 VGA –video recording (up to 30 f/s)
H=Higher quality: max 32 kB 8GB hard-disk inside!
L=Lower quality: typical 15 kB
2.27

Some Digital Cameras

Miniature Digital Camera - Keychain

640x480 Digital Camera USB

http://www.compuvisor.com/mike64
di
dicaus.html
ht l

2.28

14
 9/15/2009

One of the smallest digital cameras

The Cubik is the world's smallest megapixel digital camera. Its 1.3 million pixel CMOS captures images at 1280x1024.
Its on-board 16mb RAM stores 50 1280x1024 or 99 640x512 low-res pictures. You can even capture a 90 second movie
(no sound, though). Although not as small as the Spyz, the Cubik is small enough to fit unobtrusively into your pocket.
The Cubik also works as a webcam.
http://www.dynamism.com/cubik/index.shtml

2.29

The SpyZ Digital Camera

• The SpyZ, our original micro-digital
camera, is about the size of a Zippo
lighter,
g , and features an aluminum
chassis (blue or silver) and a loop for
a keychain. The 350,000 pixel CCD
takes 640x480 photos on internal
flash memory. (Up to 26 photos.) The
camera then connects directly to your
computer's USB port (mini-USB to
USB cable is included). While
connected, you can also use it as a
webcam for videoconferencing. In
digital camera mode, it runs on one
AAA battery; in webcam mode,
mode it
draws power from the USB port.

http://www.dynamism.com/spyz/index.sh
tml

2.30

15
 9/15/2009

Chapter 2: Digital Image Fundamentals

Principles of Image Aquisition, Sampling

and Quantization

2.31

A Simple Image Model

• Image: a 2
2-D
D light
light-intensity
intensity function f(x,y)
f(x y)

• The value of f at (x,y) Æ the intensity

(brightness) of the image at that point

• 0 < f(x,y) < ∞

32

16
 9/15/2009

A Simple Image Formation Model

Consider the monochrome case, e.g., black and white images

Represent the spectral intensity distribution of the image by a
continuous function f(x,y), i.e., for fixed value of (x,y), f(x,y) is
proportional to the grey level of the image at that point.
Of course,
(black) 0 ≤ f(x,y) ≤ fmax (white)
Why such limits?
Lower bound is because light intensity is a real positive quantity (recall
that intensity
y f is p
proportional
p to ||E||2, where E is the electric field).
)
Upper bound is due to the fact that in all practical imaging systems, the
physical system imposes some restrictions on the maximum
intensity level of an image, e.g., film saturation and cathode ray tube
phosphor heating.
Intermediate values between 0 and fmax are called shades of gray
varying from black to white. 2.33

Digital Image Acquisition

34

17
 9/15/2009

A Simple Image Model

• Nature of f(x,y):
f(x y):

– The amount of source light incident on the scene

being viewed

– The amount of light reflected by the objects in

the scene

35

A Simple Image Model

• Illumination & reflectance components:

– Illumination: i(x,y)
– Reflectance: r(x,y)

– f(x,y) = i(x,y) ⋅ r(x,y)

i(x,y)) < ∞
– 0 < i(
and 0 < r(x,y) < 1
(from total absorption to total reflectance)

36

18
 9/15/2009

A Simple Image Model

• Sample values of r(x,y):
r(x y):
– 0.01: black velvet
– 0.93: snow

• Sample values of i(x,y):

– 9000 foot-candles: sunny day
– 1000 foot-candles: cloudy day
– 0.01 foot-candles: full moon

37

A Simple Image Model

• Intensity of a monochrome image f at (xo,yyo):
gray level l of the image at that point
l=f(xo, yo)

• Lmin ≤ l ≤ Lmax
– Where Lmin: positive
Lmax: finite

38

19
 9/15/2009

A Simple Image Model

• In practice:
– Lmin = imin rmin and
– Lmax = imax rmax

• E.g. for indoor image processing:

– Lmin ≈ 10 Lmax ≈ 1000

• [Lmin, Lmax] : gray scale

– Often shifted to [0,L-1] Æ l=0: black
l=L-1: white

39

Sampling & Quantization

• The spatial and amplitude digitization of
f(x,y) is called:

– image sampling when it refers to spatial

coordinates (x,y) and

– gray-level
gray level quantization when it refers to the
amplitude.

40

20
 9/15/2009

Digital Image

41

Sampling and Quantization

42

21
 9/15/2009

A Digital Image

43

Sampling & Quantization

⎡ f (0,0) f (0,1) ... f (0, M − 1) ⎤
⎢ f (1,0) ... ... f (1, M − 1) ⎥⎥
f ( x, y ) = ⎢
⎢ ... ... ... ... ⎥
⎢ ⎥
⎣ f ( N − 1,0) f ( N − 1,1) ... f ( N − 1, M − 1)⎦

Digital Image Image Elements

(Pi l )
(Pixels)

44

22
 9/15/2009

stop

Sampling & Quantization

• Important terms for future discussion:

– Z: set of real integers

– R: set of real numbers

45

Sampling & Quantization

• Sampling: partitioning xy plane into a grid

– the coordinate of the center of each grid is a pair

of elements from the Cartesian product Z x Z (Z2)

• Z2 is the set of all ordered p

pairs of elements
(a,b) with a and b being integers from Z.

46

23
 9/15/2009

Sampling & Quantization

• f(x,y)
f(x y) is a digital image if:

– (x,y) are integers from Z2 and

– f is a function that assigns a gray-level value
(from R) to each distinct pair of coordinates (x,y)
[quantization]

• Gray levels are usually integers

– then Z replaces R

47

Sampling & Quantization

• The digitization process requires decisions
about:

– values for N,M (where N x M: the image array)

and

– the
h number
b off discrete
di gray levels
l l allowed
ll d ffor
each pixel.

48

24
 9/15/2009

49

Sampling & Quantization

• Usually
Usually, in DIP these quantities are integer
powers of two:
N=2n M=2m and G=2k

number of gray levels

• Another assumption is that the discrete

levels are equally spaced between 0 and L-1
in the gray scale.

50

25
 9/15/2009

Examples

1 MP (mega-pixel) 1/4 MP

51

Examples

52

26
 9/15/2009

Examples

8-bit 7-bit

6-bit 5-bit

53

Examples

4-bit 3-bit

2-bit 1-bit

54

27
 9/15/2009

Sampling & Quantization

• If b is the number of bits required to store a
digitized image then:

– b = N x M x k (if M=N, then b=N2k)

55

Chapter 2: Digital Image Fundamentals

The number of bits required to store an image is b=MxNxk

and when M=N,, b becomes N2k.

Ex. 8-bit images of size 1024 by 1024 and higher require

a significant storage space!

How do these parameters (N and k) affect the image? 56

28
 9/15/2009

Sampling & Quantization

• How many samples and gray levels are
required for a good approximation?

– Resolution (the degree of discernible detail) of

an image depends on the number of samples
(spatial resolution, e.g. 300 dpi) and the number
of gray levels (intensity resolution
resolution, e
e.g.
g 8-bit)
8 bit).
– i.e. the more these parameters are increased,
the closer the digitized array approximates the
original image.

57

STOP

Sampling & Quantization

• How many samples and gray levels are
required for a good approximation? (cont.)

– But: storage & processing requirements increase

rapidly as a function of N, M, and k

58

29
 9/15/2009

Sampling & Quantization

• Different versions (images) of the same
object can be generated through:

– Varying N, M numbers
– Varying k (number of bits)
– Varying both

59

Chapter 2: Digital Image Fundamentals

Example 1: Spatial Resolution: we keep k constant at 8 bits

and we varyy N from 1024 to 32.

How? The original 1024 by 1024 image is subsampled by removing every other60
column
and every other row to produce the 512 by 512 image.

30
 9/15/2009

Image Resampling:
To visualize the difference, we up-sample (by duplication)
to the original size of 1024 by 1024.

1024x1024 256x256

32 32
32x32
128x128

61

Example 2: we keep the image size constant at 452x374 and reduce the
number of gray levels L from 256 to 2 (i.e. reduce k from 8 to 1)

iin this
hi 32-level
32 l l image,
i
note the appearance
64 levels of very fine ridge-
like structures in the
areas of smooth gray
levels, e.g. skull.
62

31
 9/15/2009

Due to insufficient number of gray levels, this artifact is more visible below
and it is called false contouring.

16
levels
8
4 2

63

Sampling & Quantization

Example 3: what happens when we vary both
N and k?
Isopreference curves (in the Nk plane)
– Each point: image having values of N and k equal
to the coordinates of this point

– Points lying on an isopreference curve correspond

to images of equal subjective quality.

64

32
 9/15/2009

low level detail medium level detail high level detail

65

Chapter 2: Digital Image Fundamentals

Isopreference [Huang 1965] curves are plotted in the Nk-plane, where each
point represents an image having values of N and k equal to the coordinates of
that point.
Points lying on an isopreference curve correspond to images of equal subjective
quality.

Comments:
1. Isopreference curves tend to shift right and upward (i.e. better image quality)
2. In images with a large amount of details, only a few gray levels are needed
3. In the other two image categories, the perceived quality remained the same
in some intervals in which N was increased but k actually decreased! (more
contrast in the image is perhaps preferred by some people!)

66

33
 9/15/2009

Sampling & Quantization

• Conclusions:
– Quality of images increases as N & k increase
– Sometimes, for fixed N, the quality improved by
decreasing k (increased contrast)
– For images with large amounts of detail, few gray
levels are needed

67

Nonuniform
Sampling & Quantization
• An adaptive sampling scheme can improve the
appearance of an image, where the sampling would
consider the characteristics of the image.

– i.e. fine sampling in the neighborhood of sharp gray-level

transitions (e.g. boundaries)
– Coarse sampling in relatively smooth regions

• Considerations: boundary detection, detail content

68

34
 9/15/2009

Nonuniform
Sampling & Quantization
• Similarly: nonuniform quantization process

• In this case:
– few gray levels in the neighborhood of
boundaries
– more in
i regions
i off smooth
th gray-level
l l variations
i ti
(reducing thus false contours)

69

Contouring Effect

• If the number of quantization levels is not sufficient

sufficient,
contouring can be seen in the image.
• Contouring starts to become visible at 6 bits/pixel.
• Quantization should attempt to keep the quantization
contours below the visible level.
To reduce this effect:
• Contrast Quantization,
• Dithering.

2.70

35
 9/15/2009

iin this
hi 32-level
32 l l image,
i
note the appearance
64 levels of very fine ridge-
like structures in the
areas of smooth gray
levels, e.g. skull.
2.71

Due to insufficient number of gray levels, this artifact is more visible below
and it is called false contouring.

16 8
4 2

2.72

36
 9/15/2009

2.73

2.74

37
 9/15/2009

Aliasing in2:Digital
Chapter Images:
Digital Image Moiré Patterns
Fundamentals

The effects of aliased frequencies can be seen under the right conditions in the form
of so-called Moiré patterns.

A Moiré pattern caused by a break up of the periodicity is seen below as a 2-D sinusoidal
(aliased) waveform running in a vertical direction.

2.75

Moiré Patterns (cont’d)

Aliasing manifests itself through

high-frequency components
masquerading as low-frequency
ones.

In images, it appears as low-

frequency patterns scattered
throughout
h h the
h iimage.

These patterns are called Moiré

patterns.

2.76

38
 9/15/2009

Chapter 2: Digital Image Fundamentals

Image Zooming: NN vs Bilinear Interpolation

NN

Bili
Bilinear

2.77

Another example: http://www.dpreview.com/learn/?/key=interpolation

High Dynamic Range Imaging

Q: Can we generate a HDR image (16bpp) by a standard camera?

A: Yes, adjust the exposure and fuse multiple LDR images together
78

39
 9/15/2009

Towards Gigapixel

Mega-pel Giga-pel

Photographers and artists have manually or semi-automatically

stitched hundreds of mega-pel pictures together to demonstrate
how a giga-pel picture looks like → the power of pixels

http://triton.tpd.tno.nl/gigazoom/Delft2.htm

79

Commonly–used Terminology
Neighbors of a pixel p=(i,j)

N8(p)={(i-1,j),(i+1,j),(i,j-1),(i,j+1),
N4(p)={(i-1,j),(i+1,j),(i,j-1),(i,j+1)}
(i-1,j-1),(i-1,j+1),(i+1,j-1),(i+1,j+1)}

Adjacency

4-adjacency: p,q are 4-adjacent if p is in the set N4(q)

8-adjacency: p,q are 8-adjacent if p is in the set N8(q)
Note that if p is in N4/8(q), then q must be also in N4/8(p)
80

40
 9/15/2009

Common Distance Definitions

Euclidean distance D4 distance D8 distance

(2-norm) (city block distance)
(city-block (checkboard distance)

2 2 5 2 52 2 4 3 2 3 4 2 2 2 2 2
5 2 1 2 5 3 2 1 2 3 2 1 1 1 2
2 1 0 1 2 2 1 0 1 2 2 1 0 1 2
5 2 1 2 5 3 2 1 2 3 2 1 1 1 2
2 2 5 2 52 2 4 3 2 3 4 2 2 2 2 2

81

Block-based Processing

82

41
 9/15/2009

Chapter 2: Digital Image Fundamentals

2.83

42