? W H A T I S . . .

JPEG?
David Austin

   
The image below consists of a rectangular array of R 1.0 0.0 1.40210
3,871,488 pixels. The color of each pixel is deter-  G  =  1.0 −0.34414 −0.71414 
   

mined by red, green, and blue components, each B 1.0 1.77180 0.0
requiring one byte of computer memory. Naive-
 
Y
ly, we expect the image to require 11,614,464 ×  Cb − 128  .
 
bytes. However, the size of the JPEG file contain- Cr − 128
ing this image is only 734,268 bytes, roughly 16
times smaller. We will describe the compression Notice that the luminance contributes equally to
algorithm, developed by the Joint Photographic the three color components. To help visualize this
Experts Group (JPEG), that allows the image to be
transformation, we show the colors that result
stored so compactly.
from fixing the luminance and mixing various
chrominance values.

Cr Cr

Cb Cb
Y =0 Y = 100
The algorithm proceeds by dividing the image
Rather than specifying the red, green, and blue into 8 by 8 blocks of pixels that are independently
components of a color, it is convenient to use three processed. Here is a sample block.
different quantities: luminance Y , which is closely
related to the brightness of the color, and blue
and red chrominances Cb and Cr , which roughly
determine the hue. An invertible affine transform
translates between the two representations; for
instance, to recover the red R, green G, and blue
B components, we use

David Austin is professor of mathematics at Grand Valley The (Y , Cb , Cr ) components in our sample 8 by 8
State University. His email address is david@merganser. block are shown below; lighter regions correspond
math.gvsu.edu. to larger values.

226 Notices of the AMS Volume 55, Number 2

 luminance DCT coefficients is
16 11 10 16 24 40 51 61
 
 12 12 14 19 26 58 60 55 
 
 14 13 16 24 40 57 69 56 
 
 14 17 22 29 51 87 80 62 
Q=
 
 18 22 37 56 68 109 103 77


 24 35 55 64 81 104 113 92
 

Y Cb Cr  49

64 78 87 103 121 120 101


Notice that the luminance values produce a 72 92 95 98 112 100 103 99
grayscale version of the image. As pyscho-visual Recognizing that the luminance carries more im-
experiments show that the human eye is most sen- portant visual information, we use different ma-
sitive to the luminance values, the color transform trices for quantizing the coefficients describing
concentrates the most important information in- luminance and chrominance. Processing our sam-
to a single component. Color television uses a ple block with an intermediate value of α, the
similar color model, enabling black and white tele- quantized luminance coefficients for our sample
visions to display efficiently color images that are block are shown below.
broadcast. u
For reasons that will be explained later, we
now express the component values as a linear
7 -2 1 0 0 0 0 0
combination of cosine functions of increasing v 4 0 1 0 0 0 0 0
frequency. For instance, if Yx,y is the luminance 1 -1 0 0 0 0 0 0
in column x and row y of our block, we write
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
7 X
X 7 
(2x + 1)uπ
 0 0 0 0 0 0 0 0
Yx,y = Cu,v Fu,v cos 0 0 0 0 0 0 0 0
u=0 v=0
16
0 0 0 0 0 0 0 0
(2y + 1)vπ
 
× cos . The quantized coefficients are ordered by fol-
16
lowing the arrows so that lower frequencies appear
(The normalizing constants Cu,v need not con- first.
cern us.) The coefficients Fu,v are found by
the two-dimensional Discrete Cosine Transform
(DCT)
7 X
7
(2x + 1)uπ
X  
Fu,v = Cu,v Yx,y cos
x=0 y=0
16
(2y + 1)vπ
 
× cos
16

and efficiently computed using a version of the

The quantized coefficients for the luminance
Fast Fourier Transform.
component in our sample then produce the se-
The values of the components in most blocks
quence 7, −2, 4, 1, 0, 1, 0, 1, −1 followed by 55
do not change rapidly, and the human eye is not
zeroes. Rather than storing each zero, we simply
particularly sensitive to these changes. Therefore,
record the number of zeroes, which reduces the
the DCT coefficients corresponding to higher fre-
storage requirement significantly. Further com-
quencies will likely be small and may be ignored
pression is obtained by using a Huffman code to
without affecting our perception of the image. This
store the sequence of coefficients compactly.
observation motivates our method for quantizing The image is reconstructed by reversing this
the DCT coefficients so they may be stored as process. The quantized coefficients give approx-
integers. imate values of Fu,v . These, in turn, give the
Two ingredients are used in the quantization (Y , Cb , Cr ) components and the (R, G, B) compo-
process. The first is a parameter α, chosen by the nents. The reconstructed block is shown to the
user to control the amount of compression and right of the original.
the quality of the image. Larger values of α lead
to smaller files and lower quality images.
The second ingredient is an 8 by 8 matrix Q =
[Qu,v ] with the coefficients quantized by rounding
Fu,v /αQu,v . The Qu,v are chosen empirically and
typically have larger values for higher frequencies
so as to deemphasize those frequencies. For in-
stance, a typical matrix used for quantizing the

February 2008 Notices of the AMS 227

 The Discrete Fourier transform (DFT) may ap- the components. Now form wavelet coefficients
pear preferable to the DCT since it is more easily lx = (y2x + y2x+1 )/2
computed; however, our choice of the DCT is
hx = (y2x − y2x+1 )/2.
explained by our desire to concentrate the in-
formation in the low-frequency coefficients. For The hx are called “high-pass” coefficients, since
instance, consider the values, yx , of one component they detect high-frequency changes, while the lx
in one row of an 8 by 8 block. The DFT expresses are “low-pass” coefficients. Order them by listing
all the low-pass coefficients followed by the high-
yx as a linear combination of functions whose
pass coefficients and perform the same operation
period is 8 resulting in a periodic extension of
on the columns of wavelet coefficients to obtain
yx . The transform therefore unnecessarily tracks
blocks of coefficients:
the change between y7 and y8 = y0 , which can
lead to significant high-frequency contributions.
In the figure below, the yx values are shown in
black while the approximations given by the three
LL HL
lowest-frequency terms of the Fourier transform
are in red.
LH HH

The coefficients in the LL sub-block are ob-

tained by averaging over 2 by 2 blocks of pixels
and hence represent a lower resolution version of
the image. The other three sub-blocks describe the
changes necessary to construct the image at the
higher resolution. We iterate this process on the LL
0 1 2 3 4 5 6 7 sub-block thus storing the image at increasingly
lower resolutions.
By comparison, the DCT expresses yx as a linear
combination of functions whose period is 16 and LL

LL
HL

HL
LH HH

that are symmetric about x = 7.5. This smoothes LH

LL
HH
HL

out the appropriate extension of yx so that the LL HL

DCT requires a relatively small contribution from LH HH
high-frequency terms. Below we see the analogous
approximation given by the DCT and note the
improvement in the approximation. (These figures
appear in [2].)
LH HH

A quantization procedure detects regions where

the values do not change significantly so that
high-pass coefficients may be safely ignored.
Rather than the wavelet transform described
above, which averages two adjacent values, the
JPEG 2000 algorithm uses the Cohen-Daubechies-
Feauveau (9, 7) wavelet transform, which finds a
0 1 2 3 4 5 6 7 weighted average over nine adjacent values and
Since the 8 by 8 blocks are processed indepen- thus produces smoother images.
dently, discontinuities at the edges of the blocks The complexity of the JPEG 2000 algorithm,
compared to that of the original JPEG algorithm,
become apparent at high compression ratios. In
is an order of magnitude greater, and at low and
addition, it is often desirable to have the ability to
medium compression ratios, the quality of the im-
reconstruct efficiently the image at intermediate
ages produced by JPEG 2000 is not substantially
resolutions. These reasons and others led to the better. At very high compression ratios, however,
creation of the JPEG 2000 compression algorithm. where JPEG’s use of 8 by 8 blocks can cause the
Among other differences, JPEG 2000 replaces the image quality to deteriorate severely, JPEG 2000
DCT with a discrete wavelet transform. offers significantly better results.
The JPEG 2000 algorithm divides the image into Since JPEG 2000 asks us to work harder to pro-
larger blocks, perhaps of size 256 by 256. To illus- duce images of generally comparable quality, it is
trate the wavelet transform, fix one row of pixels not a clearly superior choice over JPEG. Indeed,
in a block and let yx represent the values of one of only a few web browsers are currently capable of

228 Notices of the AMS Volume 55, Number 2

displaying JPEG 2000 images. Its principal advan-
tage lies in providing a much more flexible format
for working with images in environments where
THE FEATURE COLUMN
the increased complexity is not problematic.
monthly essays on mathematical topics
For instance, the ability to reconstruct efficient-
ly the image at different resolutions, which results
from the use of the wavelet transform, allows
users to search visually through many images at
a low resolution quickly. JPEG 2000 also permits
regions, designated by a user perhaps, to be dis- www.ams.org/featurecolumn
played at a higher resolution, a reason that it
is commonly used in medical imagery. Finally, it
is possible for digital photographs, stored on a Each month, the Feature Column provides an
camera’s memory card in the JPEG 2000 format, online in-depth look at a mathematical topic.
to be converted efficiently to a lower resolution to Complete with graphics, links, and references,
reduce memory usage after they are taken. JPEG the columns cover a wide spectrum of math-
2000, being designed roughly ten years after JPEG, ematics and its applications, often including
also includes other features, such as the ability to
encrypt images, whose need was not anticipated
historical figures and their contributions.
earlier. The authors—David Austin, Bill Casselman,
Joe Malkevitch, and Tony Phillips—share their
excitement about developments in mathematics.
David Austin also wrote about JPEG in
“Image Compression: Seeing What’s Not
There”, the September 2007 installment of Recent essays include:
the Feature Column on the AMS website
(see http://www.ams.org/featurecolumn/ How Google Finds Your Needle in the Web's
archive/image-compression.html). The Haystack
Feature Column, which appears monthly,
presents accessible essays about mathematics Rationality and Game Theory
that are designed for those who have already
Lorenz and Modular Flows: A Visual
discovered the joys of mathematics as well
as for those who are just getting to know its Introduction
charms. The Feature Column is freely available The Princess of Polytopia: Alicia Boole Stott
at http://www.ams.org/featurecolumn.
and the 120-cell
Finite Geometries?
Further Reading Voronoi Diagrams and a Day at the Beach
[1] T. Archarya and P.-S. Tsai, JPEG2000 Standard for
Image Compression: Concepts, Algorithms and VLSI Simple Chaos – The Hénon Map
Architectures, Wiley, Hoboken, 2005.
[2] J. Blinn, What’s the Deal with the DCT?, IEEE The Octosphericon and the Cretan Maze
Computer Graphics and Applications 13 (4) (1993),
78–83. Trees: A Mathematical Tool for All Seasons
[3] Joint Photographic Experts Group, http://www.
jpeg.org/. Variations on Graph Minor
Penrose Tilings Tied up in Ribbons
Topology of Venn Diagrams

AMS members: Sign up for the AMS members-only

Headlines & Deadlines service at www.ams.org/enews
to receive email notifications when each new column is
posted.

February 2008 Notices of the AMS 229