COLOR IMAGE

PROCESSING
April 10, 2024
HUMAN VISUAL SYSTEM

▪ Visual perception: the ability to interpret

information and surroundings from the effects of
visible light reaching the eye
Also known as eyesight, sight, or vision
▪ Various physiological components involved in
vision - visual system
▪ Focus of much research in psychology, cognitive
science, neuroscience, and molecular biology

2
STRUCTURE OF THE HUMAN EYE

▪ Cornea and sclera keep everything in!

▪ Choroid contains all blood vessels that serve nutrition to eye

▪ Iris contains the pigment that gives our eyes their colour
▪ The lens focuses light from objects onto the retina
4
STRUCTURE OF THE HUMAN EYE

2 types of Receptors:
Cones (for details) and
Rods (for general vision)

▪ Muscles controlling the eye rotate the eyeball until the image
falls on the fovea
▪ Pattern vision is possible through the distribution of discrete
light receptors over the surface of the retina 5
STRUCTURE OF THE HUMAN EYE
• Cones (Cone vision: photopic or bright-light vision)
▪ 6-7 million, located primarily in the central portion of the
retina (the fovea)
▪ Highly sensitive to color; each is connected to its own
nerve end thus human can resolve fine details
• Rods (Rod vision: scotopic or dim-light vision)
▪ 75-150 million, distributed over the retina surface
▪ Several rods are connected to a single nerve end reduce
the amount of detail discernible
▪ Give a general, overall picture of the field of view
▪ Sensitive to low levels of illumination
6
STRUCTURE OF THE HUMAN EYE

▪ Blind spot: the absence of receptors area

▪ Fovea: a circular indentation in the retina of about 1.5 mm in
diameter
▪ Receptor density: measured in degrees from the fovea
▪ Cones are most dense in the center of the retina (in the area
of the fovea)
▪ Rods increase in density from the center out to approx. 20°
off axis and then decrease in density out to the extreme
periphery of the retina 7
IMAGE FORMATION IN THE EYE
• Muscles within the eye can be used to change the shape
of the lens allowing us focus on objects that are near or
far away
• An image is focused onto the retina causing rods and
cones to become excited which ultimately send signals
to the brain

8
HUMAN VISUAL SYSTEM

▪ The existence of two distinct visual systems

(based on two types of retinal cells),

Scotopic and Photopic,
results in an extraordinary dynamic range of
light intensity levels to which the human
visual system can adapt.

9
 PHYSICAL NATURE OF COLOR

• Visible energy - a small

portion of the electro-
magnetic spectrum

380 780

Pure monochromatic colors

• The colours that we perceive are determined by

the nature of the light reflected from an object

18
 COLOR FUNDAMENTALS
▪ Color in the eye
o Varying sensitivity of different cells
in the retina (cones) to light of
different wavelengths:
• S-cones: short-wavelength (blue);
• M-cones: middle-wavelength (green);
• L-cones: long-wavelength (red).

Normalized typical human cone

cell responses (S, M, and L
types) to monochromatic
spectral stimuli.
The difference in the signals Rod cells have a peak sensitivity at
received from the three cone 498 nm, roughly halfway between the
types allows the brain to perceive peak sensitivities of the S and M cones.
all possible colors. 20
 COLOR FUNDAMENTALS
▪ Color in the eye
o The color yellow is perceived when the L
cones are stimulated slightly more than
the M cones.
o The color red is perceived when the L
cones are stimulated significantly more
than the M cones.
o Blue and violet hues are perceived when
the S receptor is stimulated more than the
other two.

At moderate to bright light levels where the cones function, the

eye is more sensitive to yellowish-green light than other colors
because this stimulates the two most common (M and L) cones
almost equally.
At lower light levels, where only the rod cells function, the
sensitivity is greatest at a blueish-green wavelength.
21
COLOR FUNDAMENTALS
▪ Approx 65% of all cones are sensitive to red light, 33% are
sensitive to green light, and only about 2% are sensitive to
blue (but the blue cones are the most sensitive)
▪ The eye can perceive other colors as a combination of
several pure colors.
▪ Most colors may be obtained as a combination of a small
number of primaries.
o Tri-stimulus Color
Theory: Any color can
be matched by a
mixture of three fixed
580 520 700 base colors (primaries)
(yellow) (green) (red)

22
 COLOR FUNDAMENTALS
▪ Primary, Secondary and Complementary colors
o Due to different absorption curves of the cones,
colors are seen as variable combinations of the so-
called primary colors: red, green and blue.
o Primary colors added to produce the secondary
colors : magenta (R+B), cyan (G+B), yellow (R+G).
Mixture of lights
(Additive primaries)o Subtractive primaries: used mainly in color
printing, where light is absorbed by dyes
o For pigments and colorants, a primary color is the
one that subtracts (absorbs) a primary color of
light and reflects the other two.

Mixture of pigments o Two color sources combined to produce white

(Subtractive primaries) color are referred as complementary colors: (R,C),
(G, M) and (B, Y) 23
 COLOR FUNDAMENTALS
▪ Hue, Brightness, and Saturation (the characteristics
generally used to distinguish one color from another)
− Hue represents the impression related to the dominant
wavelength of the color stimulus
− Dominant color as perceived by an observer
− When we call an object red/yellow, we are referring to its hue

− Brightness (synonym of intensity) is perceived intensity

− Radiant energy emitted per unit time, per unit solid angle, and
per unit projected area of the source (luminance)

− Saturation expresses the relative color purity

− amount of white light in the color
− The pure spectrum colors are fully saturated

− Hue and Saturation - the chromaticity coordinates

24
 COLOR FUNDAMENTALS
▪ No finite set of real primary colors can be combined
to produce all possible visible colors
• Color matching functions obtained
by averaging the judgments of a
large number of observers
• Colors in the vicinity of 500nm can
only be matched by ‘subtracting’
an amount of red light from a
combination of blue and green
lights
Amounts of red, green, & • An RGB color monitor cannot
blue color needed to display colors in the neighborhood
produce any spectral color of 500nm

Three primaries are sufficient for most purposes 25

 COLOR FUNDAMENTALS
▪ Three standard primaries were defined in 1931 by
Commission Internationale de l´Éclairage (CIE)
▪ The three standard primaries are imaginary colors

Defined X, Y, and Z primaries to replace red, o Defined mathematically

green and blue primaries with +ve color matching
They are defined tabularly at 1 functions that specify
Color Matching CIE Amounts

nm intervals for color samples

that subtend 2° field of view on the amount of each
retina
primary needed to
describe any spectral
color

o This provides an
international standard
definition for all colors
26
 COLOR FUNDAMENTALS
▪ From tristimulus values to chromaticity coordinates
o The amounts of R, G, and B needed to form any particular
color are called the tristimulus values and denoted by X, Y,
and Z (imaginary colors)
o A color is specified as a vector in 3D space by its
trichromatic coefficients: Cλ = XX+YY+ZZ
(X,Y and Z: amount of standard primaries needed to match Cλ)

o Normalize the amounts against luminance (X+Y+Z)

X Y Z
x=
X +Y + Z
, y=
X +Y + Z
, z=
X +Y + Z
with x + y + z = 1
Provides a way to find the chromaticity coordinates x and y
(Cartesian system). A complete description of color given with x, y,
and Y. x z
X = Y , and Z = Y
27
y y
 COLOR FUNDAMENTALS
▪ Standard primaries and the CIE xy Chromaticity
Diagram
o Color composition as a function of x
The marked point has approx. 62% green and (red) and y (green)
25% red content (blue = approx 13%)
o The corresponding value of z (blue):
z = 1 – (x + y)
o Outer boundary is the spectral
* (monochromatic) locus (pure colors),
wavelengths shown in nm.
c o Purple line: not part of the spectrum

Purple o (x,y) = (1/3,1/3) is a flat energy

line spectrum point (point of equal energy).
o Any point on the boundary is completely
Plot of normalized amounts saturated (Boundary → point of equal
x and y for colors in visible
spectrum energy : saturation → 0) 28
CIE CHROMATICITY DIAGRAM
Uses of CIE Chromaticity Diagram:
o Name colors
ideal green o Identifying complementary colors

520 nm o Define color mixing

o Determining dominant wavelength
540 nm
510 nm
560 nm and purity of a given color
green o Define and compare color gamuts for
500 nm 580 nm different set of primaries

yellow 600 nm
cyan white 700 nm
490 nm red
blue

400 nm ideal red 29

ideal blue
 COLOR FUNDAMENTALS
• Chromaticity Diagram: useful for color mixing
o A straight line segment
joining any two points 510
520
540
B
in the diagram defines 560

all the different color 500

580

600
variations that can be 490 700

obtained by
R
combining these two 400 G
colors additively
Three primaries can
o A line drawn from the only generate colors
point of equal energy inside or on the
to any point on the
The CIE 1931 bounding edges of
boundary of the chart:
chromaticity diagram the triangle
all shades of that
particular spectrum 30
color
 COLOR FUNDAMENTALS
• Chromaticity Diagram: useful for color mixing

o The chromaticity diagram helps

us understand why no set of three
primaries can be additively
RGB monitor
color gamut combined to generate all colors

printing device
o Color gamuts for video monitors
color gamut
and hard copy devices are
conveniently compared on the
chromaticity diagram
o The Color Gamuts of different
Typical gamuts of a displays and printers are not
monitor and of a likely to match. Printers usually
printing device have smaller gamuts. 31
 COLOR MODELS
▪ The color of an object depends not only on the object itself,
but also on
▪ Light source illuminating it;
▪ Color of the surrounding area; and
▪ The human visual system

▪ A color model (or color space or color system) facilitates

the specification of colors in some standard way.
▪ Artists often specify color as different Tints ‘Pure’
tints, shades, and tones of strongly color
saturated or pure, pigments Grays Tones
Shades

32
 COLOR MODELS
• Provides a coordinate system and a subspace in it
where each color is represented by a single point.
• No single color model can explain all aspects of color
• Different models are used to help describe the
different perceived characteristics of color
• Common Color Models:
− RGB (monitors, video cameras),
− CMY/CMYK (printers), h/w oriented
models
− HSI/HSV/HSL/HSB (image processing),
− CIE Lab (image processing). Device independent model
33
COLOR MODELS

▪ RGB color model

If R,G, and B are represented with

8 bits (24-bit RGB image), the
total number of colors is
(28)3=16,777,216

The main diagonal of the RGB cube represents all shades of gray. 34
 COLOR MODELS

▪ RGB color model

• Set of safe RGB colors/Set of all systems-safe colors

• Assumption: min 256 colors can be reproduced faithfully
by any system
• Accepted standard notation to refer these colors
• 40/256 colors are known to be processed differently by
various OS
• Leaves only 216 colors common to most systems (de
facto standard for safe colors in Internet applications)
• They are used whenever it is desired that the colors
viewed by most people appear the same
35
COLOR MODELS

▪ RGB color model

o Set of safe RGB colors/Set of all systems-safe colors

o Each of the 216 safe colors – formed from 3 RGB values
– but each value can only be 0, 51, 102, 153, 204, or 255
o RGB triplets of these values give us (6)3=216 possible
values
o It is customary to express these values in the hexagonal
number system

Purest red: FF0000

Black: 000000
White: FFFFFF
36
 COLOR MODELS
▪ RGB safe color cube
The 216
safe RGB
colors and
all the
grays that
are part of
the safe
Unlike the full-color cube which color
is solid, this cube has valid group
colors only on the surface planes

Top left array, 1st row : FFFFFF, FFFFCC, FFFF99 and so on

Top left array, 2nd row : FFCCFF, FFCCCC, FFCC99 …(Final square: FF0000)
Second array starts with value CCFFFF and proceed in the same manner

37
COLOR MODELS
▪ CMY color model
Y o The Cyan, Magenta and Yellow primaries are the
G complements of Red, Green and Blue respectively
W R o When used as filters to subtract color from white
C B light, they are called subtractive primaries
M o When a surface is coated with cyan ink, no red
light is reflected from the surface

Red = White-Cyan = White-Green-Blue (0,1,1)

Green = White-Magenta = White-Red-Blue (1,0,1)
Blue = White-Yellow = White-Red-Green (1,1,0)
o Devices that deposit colored pigments 𝐶 1 𝑅
on paper (printers & copiers), require 𝑀 = 1- 𝐺
CMY data input or perform an RGB to
𝑌 1 𝐵
CMY conversion internally 38
COLOR MODELS
▪ Conversion from CMY to CMYK color model

39
COLOR MODELS

▪ HSI/HSL/HSV/HSB color spaces

− RGB, CMY/CMYK are hardware-oriented color

spaces (suited for image acquisition and display).

− (Hue, Saturation, Intensity/Lightness/ ValuThe

HSI/… e/Brightness) are perceptive color spaces
(suited for image description and interpretation).

− have more intuitive appeal to a user

− Allow the decoupling of chromatic signals (H+S)

from the intensity signal (I).

40
COLOR MODELS
• HSI color space V

green yellow
120˚

cyan 1.0 red

0˚
blue
240˚ magenta

H
black S
0.0

41
COLOR MODELS
▪ HSI color space
o Human eye can distinguish about
128 different hues and about 130
saturation levels
o For each of these, a number of
value settings can be detected,
depending on the hue selected
o 23 shades with yellow color
o 16 shades with blue color
o For most graphics applications,
128 hues, 8 saturation levels and
15 value settings are sufficient
Graphical depiction of HSV
(cylinder and cone) o 16384 available colors, 14 bits of
color storage per pixel
42
COLOR MODELS
▪ Conversion from RGB to HSI
Given an image in RGB color format, the H component of each
RGB pixel is:
𝜃 𝑖𝑓 𝐵 ≤ 𝐺
𝐻=ቊ
360 − 𝜃 𝑖𝑓 𝐵 ≤ 𝐺
1
𝑅−𝐺 + 𝑅−𝐵
with 𝜃 = cos −1 2
1ൗ
𝑅−𝐺 2 + 𝑅−𝐵 𝐺−𝐵 2

The saturation component is given by:

3
𝑆 =1− min 𝑅, 𝐺, 𝐵
𝑅+𝐺+𝐵
Finally, the intensity component is given by:
1
𝐼 = 𝑅+𝐺+𝐵
3 43
COLOR MODELS
▪ Conversion from RGB to HSI
𝜃 𝑖𝑓 𝐵 ≤ 𝐺
𝐻=ቊ
360 − 𝜃 𝑖𝑓 𝐵 ≤ 𝐺
1
𝑅−𝐺 + 𝑅−𝐵
with 𝜃 = cos −1 2
1ൗ
𝑅−𝐺 2 + 𝑅−𝐵 𝐺−𝐵 2

3
𝑆 =1− min 𝑅, 𝐺, 𝐵
𝑅+𝐺+𝐵
1
𝐼 = 𝑅+𝐺+𝐵
3
Assumption: RGB values normalized in the range [0,1]
Angle θ is measured wrt the red axis
Hue can be normalized to the range [0,1] by dividing by 3600
44
COLOR MODELS
▪ Conversion from HSI to RGB
Given values of HIS in the interval [0, 1]
Applicable equations depend on the value of H
(multiply H by 3600, gives hue to original range of [00,
3600])
RG sector (00 ≤ H< 1200):
When H is in this sector, the RGB components are given by
𝐵 =𝐼 1−𝑆
𝑆 cos 𝐻
𝑅 =𝐼 1+
cos 600 − 𝐻
𝐺 = 3𝐼 − (𝑅 + 𝐵)

45
COLOR MODELS
▪ Conversion from HSI to RGB
RG sector (1200 ≤ H< 2400):
𝐻 = 𝐻 − 1200
𝑅 =𝐼 1−𝑆
𝑆 cos 𝐻
𝐺 =𝐼 1+
cos 600 − 𝐻
𝐵 = 3𝐼 − (𝑅 + 𝐺)
RG sector (2400 ≤ H< 3600):
𝐻 = 𝐻 − 2400
𝐺 =𝐼 1−𝑆
𝑆 cos 𝐻
𝐵 =𝐼 1+
cos 600 − 𝐻
𝑅 = 3𝐼 − (𝐺 + 𝐵) 46
COLOR IMAGE PROCESSING
▪ Basics
o Full-color images have at least three components,
therefore color pixels are vectors

o Two approaches:
1. Process each component image individually and
then form a composite processed color image from
the individually processed components
2. Work with color pixels directly
o The results of individual color component processing are not
always equivalent to direct processing in color vector space,
in which case we must formulate new approaches 47
COLOR IMAGE PROCESSING
▪ Basics
o Neighborhood spatial processing of gray-scale
and full-color images:

Averaging done on
a per-color-
component basis
and then the vector
is formed 48
COLOR IMAGE PROCESSING
▪ Color Transformations - intensity transformation
o 𝑔 𝑥, 𝑦 = 𝑇 𝑓 𝑥, 𝑦
o The pixel values here are triplets or quartets from the color
space chosen to represent the images
o Basic intensity transformation:
𝑠𝑖 = 𝑇𝑖 𝑟1 , 𝑟2 , … . , 𝑟𝑛 , 𝑖 = 1, 2, … , 𝑛
where 𝑠𝑖 and 𝑟𝑖 denote the color components of 𝑓 𝑥, 𝑦 and
𝑔 𝑥, 𝑦 at any point 𝑥, 𝑦 ;
n is the no. of color components; and
𝑇1 , 𝑇2 , … 𝑇𝑛 is a set of transformation or color mapping
functions that operate on 𝑟𝑖 to produce 𝑠𝑖
value of 𝑛 comes from the color space chosen to describe 49
images
COLOR IMAGE PROCESSING
▪ Color Transformations - intensity transformation

50
COLOR IMAGE PROCESSING
▪ Color Transformations - intensity transformation
o Modifying the intensity of the full-color image using
𝑔 𝑥, 𝑦 = 𝑘𝑓 𝑥, 𝑦 where 0 < k < 1.
o This transformation in HSI color space:
𝑠3 = 𝑘𝑟3 ; 𝑠1 = 𝑟1 ; 𝑠2 = 𝑟2 (only intensity component is modified)
o This transformation in RGB color space:
𝑠𝑖 = 𝑘𝑟𝑖 ; i=1,2, 3
o This transformation in CMY color space:
𝑠𝑖 = 𝑘𝑟𝑖 + 1 − 𝑘 ; i=1,2, 3
o HIS transformation involves the fewest no. of operations, the
computations required to convert an RGB or CMY(K) image to the HIS
space more than offsets the advantages of the simpler transformation
51
 COLOR IMAGE PROCESSING
• Color Transformations - intensity transformation
o Result of applying intensity transformation discussed to
the full color image

52
COLOR IMAGE PROCESSING
▪ Color Complements: useful for enhancing detail that is
embedded in dark regions of a color image

In HSI model, saturation component of

the complement cannot be computed
from the saturation component of the
input image alone
Note that the saturation component of the input image is unaltered and
responsible for the visual differences between fig (c) and (d) 53
 DEVICE INDEPENDENT COLOR MODELS
▪ Nonlinear color space
o Problem with chromaticity diagram: it is highly non-linear, in that a
vector of unit magnitude representing the difference between two
chromaticities is not uniformly visible.
o Desirable: To produce a chromaticity diagrams in which a vector
of unit magnitude (difference between two points representing
two colors) is equally visible at all colors.
o Perceptually uniform means that a change of the same amount in a
color value should produce a change of about the same visual
importance.
o Attempt to linearize the perceptibility of unit vector color
differences (model of choice for many color management systems
(CMS))
o CIE L*u*v* 58
o CIE L*a*b*
 DEVICE INDEPENDENT COLOR MODELS
▪ CIE L*a*b* color space
o based directly on CIE XYZ
o device independent and perceptually uniform color
model
o Intended to provide a standard, approximately uniform
color space which could be used by everyone so that
color values could be easily compared
o Organized in the cube form
o The L* axis runs from top to bottom
o Max value 100, represents a perfect reflecting diffuser
o Minimum 0, represents black

o The a* and b* axes have no specific numerical limits 59

 DEVICE INDEPENDENT COLOR MODELS
▪ CIE L*a*b* color space

L*= 25

L*= 75

o The non-linear relationships for L* a* and b* are

intended to mimic the logarithmic response of the eye. 60
 DEVICE INDEPENDENT COLOR MODELS
o X, Y, Z are the CIE
▪ CIE L*a*b color space Tristimulus values
o The L*a*b* components are given o Coloring information is
by referred to the color of
the white point of the
system, subscript 𝑛
o Typical standard: D65
o Like HIS, the L*a*b
system is an excellent
decoupler of intensity
(denoted by L*) and
color
o This makes it useful in
Check http://www.hunterlab.com/appnotes/an07_96a.pdf image tone & contrast
and http://www.hunterlab.com/appnotes/an10_07.pdf editing and compression
61
applications
 COLOR IMAGE PROCESSING
▪ Tone and Color Corrections
o Calibrated imaging systems allow tonal and color
imbalances to be corrected interactively and independently
o First, image tonal range is corrected and then color
irregularities, like over- and under-saturated colors are
resolved
o Tonal range of an image (key type): general distribution of
color intensities
o The colors of high-key (low-key) images is concentrated at
high intensities (low intensities). Middle-key images lie in
between.
o Desirable: distribute the intensities of a color image equally
between the highlights and the shadows
62
62
 COLOR IMAGE PROCESSING
▪ Tone and Color Corrections: Examples
o The idea is to adjust
experimentally the image’s
brightness and contrast to
provide maximum detail over a
suitable range of intensities
o In RGB and CMY(K) spaces:
same transformation functions
for all color components
o In HSI spaces: only intensity
component modified
Tonal corrections for flat, high-key, and
low-key color images (adjusting RGB
components equally does not always
alter the image hues significantly) 63
63
 COLOR IMAGE PROCESSING
▪ Tone and Color Corrections: Examples

o Variety of ways to correct

color imbalance
o Adjusting color
components affacts the
overall color balance of
the image
o Use of color wheel to
predict how one color
component will affect
others

64
64
 COLOR IMAGE PROCESSING
▪ Histogram Processing

o Usually histogram
equalization of the
components of a
color image
independently results
in erroneous color
o More logical: spread
the color intensities
uniformly , leaving
the colors unchanged
o HIS color space
ideally suited for this
65
COLOR IMAGE PROCESSING
▪ Color Image Smoothing
o The concept of spatial filtering operation is easily
extended to the processing of full-color images – deal
with the component vectors

66
COLOR IMAGE PROCESSING
▪ Color Image Sharpening using the Laplacian
o The Laplacian of a vector is defined as a vector whose
components are equal to the Laplacian of the
individual scale components of the input vector

67
 REFERENCES
o Gonzalez R, Woods R. Digital Image Processing, published by Pearson
Education, Inc, 2002

o Hearn and Baker, Computer Graphics, published by Pearson

Education, Inc, 2005

o Some slides by Natalia Vassilieva and Content-based Image Retrieval

Methods. In Programming and Computer Software, 2009,Vol. 35, No. 3,
pp. 158–180.

68