Image Compression

CS474/674 – Prof. Bebis

Chapter 8 (except Sections 8.10-8.12)
 Image Compression
• The goal of image compression is to reduce the amount
of data required to represent a digital image.
 Image Compression (cont’d)

• Lossless
– Information preserving
– Low compression ratios

• Lossy
– Information loss
– High compression ratios

Trade-off: information loss vs compression ratio

 Data ≠ Information

• Data and information are not synonymous terms!

• Data is the means by which information is conveyed.

Goal of data compression

Reduce the amount of data while preserving as much

information as possible!
 Data vs Information (cont’d)
• The same information can be represented by different
amount of data – for example:

Ex1: Your wife, Helen, will meet you at Logan Airport

in Boston at 5 minutes past 6:00 pm tomorrow
night

Ex2: Your wife will meet you at Logan Airport at 5

minutes past 6:00 pm tomorrow night

Ex3: Helen will meet you at Logan at 6:00 pm

tomorrow night
 Compression Ratio

compression

Compression ratio:
Relevant Data Redundancy

Example:
 Types of Data Redundancy

(1) Coding Redundancy

(2) Interpixel Redundancy
(3) Psychovisual Redundancy

• Data compression attempts to reduce one or more of

these redundancy types.
 Coding Redundancy
To reduce coding redundancy, we need efficient
coding schemes!
• Code: a list of symbols
(letters, numbers, bits etc.)
• Code word: a sequence of
symbols used to represent
some information (e.g.,
gray levels).
• Code word length: number
of symbols in a code word.
– Could be fixed or variable
 Coding Redundancy (cont’d)
• To compare the efficiency of different coding schemes,
we need to compute the average number of symbols
Lavg per code word.

N x M image
rk: k-th gray level
Example: l(rk): # of bits for rk
P(rk): probability of rk

Average Image Size: E ( X )   xP ( X  x)

x
 Coding Redundancy (cont’d)
• Case 1: l(rk) = fixed length (code 1)

Example:
 Coding Redundancy (cont’d)
• Case 2: l(rk) = variable length (code 2 – Huffman code)

Total number of bits: 2.7NM

 Interpixel redundancy
• Interpixel redundancy implies that pixel values are
correlated (i.e., a pixel value can be reasonably predicted
by its neighbors).
histograms

auto-correlation


f ( x) o g ( x )   f ( x ) g ( x  a )da


auto-correlation: f(x)=g(x)
 Interpixel redundancy (cont’d)
• To reduce interpixel redundancy, some kind of a
transformation must be applied on the data.
Example Additional savings using
run-length coding

Original
threshold

threshold

11 ……………0000……………………..11…..000…..

Binary image

(1+10) bits/pair
 Psychovisual redundancy
• The human eye is more sensitive to the lower frequencies
than to the higher frequencies in the visual spectrum.
• Idea: discard data that is perceptually insignificant!
256 gray levels 16 gray levels 16 gray levels + random noise

i.e., add a small

Example: pseudo-random
number
quantization to each pixel
prior to
quantization

C=8/4 = 2:1
 Measuring Information

• The key question in image compression is:

“What is the minimum amount of data that is

sufficient to describe completely an image
without loss of information?”

• How do we measure the information content of an

image?
 Measuring Information (cont’d)
• We assume that information generation is a
probabilistic process.

• Associate information with probability!

A random event E with probability P(E) contains:

Note: I(E)=0 when P(E)=1

 How much information
does a pixel value contain?

• Suppose that gray level values are generated by a

random process, then rk contains:

units of information!

(assuming statistically independent random events)

 How much information does an image contain?

• Average information content of an image:

L 1
E   I (rk ) P(rk )
k 0

using

units/pixel
Entropy:
(e.g., bits/pixel)
 Redundancy

• Redundancy:
(data vs info)

where:

Note: if Lavg= H, then R=0 (no redundancy)

 Entropy Estimation
• It is not easy to estimate H reliably!

image
 Entropy Estimation (cont’d)
• First order estimate of H:

• What is the redundancy?

R= Lavg- H where Lavg = 8 bits/pixel

R= 6.19 bits/pixel
 Estimating Entropy (cont’d)
• Second order estimate of H:
– Use relative frequencies of pixel blocks :

image
 Estimating Entropy (cont’d)

• What does it mean that the first- and second-order

entropies are different?

• In general, differences between first-order and

higher-order entropy estimates indicate the presence
of interpixel redundancy (i.e., need to apply some
transformation).
Differences in Entropy Estimates (cont’d)
Example:
take pixel differences

16
Differences in Entropy Estimates (cont’d)
Example (cont’d):

– What is the entropy of the pixel differences image?

(better than the entropy of the original image H=1.81)

– An even better transformation should be possible since the

second order entropy estimate is lower:
 Image Compression Model

We will focus on the Source Encoder/Decoder only.

 Encoder

• Mapper: transforms data to account for interpixel redundancies.

 Encoder (cont’d)

• Quantizer: quantizes the data to account for psychovisual

redundancies.
 Encoder (cont’d)

• Symbol encoder: encodes the data to account for coding

redundancies.
 Decoder

• The decoder applies the inverse steps.

• Note that the quantization is irreversible in general!

 Fidelity Criteria

• How close is to ?

• Criteria
– Subjective: based on human observers
– Objective: mathematically defined criteria
Subjective Fidelity Criteria
 Objective Fidelity Criteria

• Root mean square error (RMS)

• Mean-square signal-to-noise ratio (SNR)

Lossless Compression
Taxonomy of Lossless Methods

(Run-length encoding)

(see “Image Compression Techniques” paper)

 Huffman Coding
(addresses coding redundancy)
• A variable-length coding technique.

• Source symbols are encoded one at a time!

• There is a one-to-one correspondence between source symbols
and code words.
• Optimal code - minimizes code word length per source
symbol.
 Huffman Coding (cont’d)
• Forward Pass
1. Sort probabilities per symbol
2. Combine the lowest two probabilities
3. Repeat Step2 until only two probabilities remain.
 Huffman Coding (cont’d)

• Backward Pass
Assign code symbols going backwards
 Huffman Coding (cont’d)
• Lavg assuming binary coding:

• Lavg assuming Huffman coding:

 Huffman Coding-Decoding

• Both coding and decoding can be implemented

using a look-up table.
• Note that decoding can be done unambiguously.
 Arithmetic (or Range) Coding
(addresses coding redundancy)

• Huffman coding encodes source symbols one at a

time which might not be efficient.
• Arithmetic coding encodes sequences of source
symbols to variable length code words.
– There is no one-to-one correspondence between source
symbols and code words.
– Slower than Huffman coding but can achieve better
compression.
 Arithmetic Coding (cont’d)

• Main idea:
– Map a sequence of symbols to a number (arithmetic code) in
the interval [0, 1).
– Encoding the arithmetic code is more efficient.

α1 α2 α3 α3 α4

– The mapping depends on the probabilities of the symbols.

– The mapping is built as each symbol arrives.
 Arithmetic Coding (cont’d)
• Main idea: α1 α2 α3 α3 α4
– Start with the interval [0, 1)

0 1
– A sub-interval of [0,1) is chosen to represent the first symbol (based on its
probability of occurrence).

0 1

– As more symbols are encoded, the sub-interval gets smaller and smaller.

0 1
– At the end, the symbol sequence is encoded by a number within the final
interval.
 Example

Encode
α1 α 2 α 3 α 3 α 4

[0.06752, 0.0688) 0.8

0.4

code: 0.068
0.2
(any number within sub-interval)

Warning: finite precision arithmetic might cause problems due to truncations!

 Example (cont’d)
• The arithmetic code 0.068 can be encoded using Binary Fraction:

0.0068 ≈ 0.000100011 (9 bits) (subject to conversion error;

exact value is 0.068359375)

• Huffman Code:
0100011001 (10 bits)

• Fixed Binary Code:

5 x 8 bits/symbol = 40 bits
α1 α2 α3 α3 α 4
 Arithmetic Decoding
1.0 0.8 0.72 0.592 0.5728

α4
0.8 0.72 0.688 0.5856 0.57152

Decode 0.572
α3
0.4 0.56 0.624 0.5728 0.56896

α2 α3 α3 α1 α2 α 4
0.2 0.48 0.592 0.5664 0.56768

α1
0.0 0.4
0.56 0.56 0.5664
 LZW Coding
(addresses interpixel redundancy)

• Requires no prior knowledge of symbol probabilities.

• Assigns sequences of source symbols to fixed length

code words.
– There is no one-to-one correspondence between source symbols
and code words.

• Included in GIF, TIFF and PDF file formats

 LZW Coding
• A codebook (or dictionary) needs to be constructed.

• Initially, the first 256 entries of the dictionary are assigned

to the gray levels 0,1,2,..,255 (i.e., assuming 8 bits/pixel)

Initial Dictionary
Dictionary Location Entry

0 0
1 1
. .
255 255
256 -

511 -
 LZW Coding (cont’d)
Example:
As the encoder examines image
39 39 126 126
pixels, gray level sequences
39 39 126 126
(i.e., blocks) that are not in the
39 39 126 126
dictionary are assigned to a new
39 39 126 126
entry.
Dictionary Location Entry

0 0
1 1
- Is 39 in the dictionary……..Yes
. . - What about 39-39………….No
255 255
256 - 39-39
* Add 39-39 at location 256
511 -
 Example
39 39 126 126 Concatenated Sequence: CS = CR + P
39 39 126 126
(CR) (P)
39 39 126 126
39 39 126 126
CR = empty
repeat
P=next pixel
CS=CR + P
If CS is found:
(1) No Output
(2) CR=CS
else:
(1) Output D(CR)
(2) Add CS to D
(3) CR=P 10 x 9 bits/symbol = 90 bits vs 16 x 8 bits/symbol = 128 bits
 Decoding LZW

• Decoding can be done using the dictionary again.

• No need to transmit the dictionary for decoding; it can
be built on the “fly” by the decoder as it reads the
received code words.
 Run-length coding (RLC)
(addresses interpixel redundancy)

• Reduce the size of a repeating string of symbols (i.e., runs):

1 1 1 1 1 0 0 0 0 0 0 1  (1,5) (0, 6) (1, 1)

a a a b b b b b b c c  (a,3) (b, 6) (c, 2)

• Encodes a run of symbols into two bytes: (symbol, count)

• Can compress any type of data but cannot achieve high
compression ratios compared to other compression
methods.
 Combining Huffman Coding
with Run-length Coding
• Assuming that a message has been encoded using
Huffman coding, additional compression can be
achieved using run-length coding.

e.g., (0,1)(1,1)(0,1)(1,0)(0,2)(1,4)(0,2)
 Bit-plane coding
(addresses interpixel redundancy)

• Process each bit plane individually.

(1) Decompose an image into a series of binary images.

(2) Compress each binary image (e.g., using run-length coding)
Lossy Methods - Taxonomy

(see “Image Compression Techniques” paper)

 Lossy Compression
• Transform the image into some other domain to reduce
interpixel redundancy.
Example: Fourier Transform

Note that the

magnitude of the FT
decreases, as u, v
increase!

K << N
K-1 K-1
 Transform Selection

• T(u,v) can be computed using various transformations,

for example:
– DFT
– DCT (Discrete Cosine Transform)
– KLT (Karhunen-Loeve Transformation) or Principal
Component Analysis (PCA)

• JPEG uses DCT for handling interpixel redundancy.

 DCT (Discrete Cosine Transform)

Forward:

Inverse:

if u=0 if v=0
if u>0 if v>0
 DCT (cont’d)

• Basis functions for a 4x4 image (i.e., cosines of

different frequencies).
 DCT (cont’d)
DFT WHT DCT

Using
8 x 8 sub-images
yields 64 coefficients
per sub-image.

Reconstructed images
by truncating
50% of the
coefficients

DCT is a more compact

transformation!

RMS error: 2.32 1.78 1.13

 DCT (cont’d)

• Sub-image size selection:

Reconstructions (75% truncation of coefficients)

original 2 x 2 sub-images 4 x 4 sub-images 8 x 8 sub-images

 DCT (cont’d)
• DCT minimizes "blocking artifacts" (i.e., boundaries
between subimages do not become very visible).

DFT
has n-point periodicity

DCT
has 2n-point periodicity
 JPEG Compression

Entropy
encoder

Accepted as
an
international
image
compression
standard in
1992.
Entropy
decoder
 JPEG - Steps

1. Divide image into 8x8 subimages.

For each subimage do:

2. Shift the gray-levels in the range [-128, 127]

3. Apply DCT  64 coefficients

1 DC coefficient: F(0,0)
63 AC coefficients: F(u,v)
 Example

[-128, 127] (DCT spectrum)

The low frequency
components are around the
upper-left corner of the
spectrum.
 JPEG Steps

4. Quantize the coefficients (i.e., reduce the amplitude of

coefficients that do not contribute a lot).

Q(u,v): quantization table

 Example

• Quantization Table Q[i][j]

Example (cont’d)

Quantization
 JPEG Steps (cont’d)

5. Order the coefficients using zig-zag ordering

- Creates long runs of zeros (i.e., ideal for run-length encoding)
 JPEG Steps (cont’d)

6. Encode coefficients:

6.1 Form “intermediate” symbol sequence.

6.2 Encode “intermediate” symbol sequence into

a binary sequence.

Note: DC coefficient is encoded differently from AC

coefficients
Intermediate Symbol Sequence – DC coeff

symbol_1 (SIZE) symbol_2 (AMPLITUDE)

(6) (61)

SIZE: # bits need to encode

the coefficient
 DC Coefficient Encoding

symbol_1 symbol_2
(SIZE) (AMPLITUDE)

predictive
coding:

The DC coefficient is substituted

by the difference between the DC
coefficient of the current block and
that of the previous block.
Intermediate Symbol Sequence – AC coeff

symbol_1 (RUN-LENGTH, SIZE) symbol_2 (AMPLITUDE) end of block

RUN-LENGTH: run of zeros preceding coefficient

SIZE: # bits for encoding the amplitude of coefficient
Note: If RUN-LENGTH > 15, use symbol (15,0) ,
 Example: AC Coefficients Encoding
Symbol_1 Symbol_2
(Variable Length Code (VLC) (Variable Length Integer (VLI)
pre-computed Huffman codes) pre-computed codes)

# bits

Smaller (and more

common) values use
fewer bytes and take
up less space than
larger (and less
common) values.

DC coefficients
(1,4) (12)  (111110110 1100) are encoded in a
VLC VLI similar way.
Final Symbol Sequence
 What is the effect of the “Quality”
parameter?

(58k bytes) (21k bytes) (8k bytes)

lower compression higher compression

What is the effect of the “Quality”
parameter? (cont’d)
Effect of Quantization:
homogeneous 8 x 8 block
 Effect of Quantization:
homogeneous 8 x 8 block (cont’d)

Quantized coeff De-quantized coeff

Effect of Quantization:
homogeneous 8 x 8 block (cont’d)

Reconstructed
Error is low!

Original
Effect of Quantization:
non-homogeneous 8 x 8 block
 Effect of Quantization:
non-homogeneous 8 x 8 block (cont’d)

Quantized coeff De-quantized coeff

 Effect of Quantization:
non-homogeneous 8 x 8 block (cont’d)

Reconstructed

Error is high!

Original
 Case Study: Fingerprint Compression

• FBI is digitizing fingerprints at 500 dots per inch with

8 bits of grayscale resolution.
• A single fingerprint card turns into about 10 MB of
data!

A sample fingerprint image

768 x 768 pixels =589,824 bytes
 WSQ Fingerprint Compression

• An image coding standard for digitized fingerprints

employing the Discrete Wavelet Transform
(Wavelet/Scalar Quantization or WSQ).

• Developed and maintained by:

– FBI
– Los Alamos National Lab (LANL)
– National Institute for Standards and Technology (NIST)
Need to Preserve Fingerprint Details

The "white" spots in the middle of

the black ridges are sweat pores and
they are admissible points of
identification in court.

These details are just a couple

pixels wide!
 What compression scheme should be used?

• Lossless or lossy compression?

• In practice lossless compression methods haven’t

done better than 2:1 on fingerprints!

• Does JPEG work well for fingerprint compression?

 Results using JPEG compression
file size 45853 bytes
compression ratio: 12.9

Fine details have been lost.

Image has an artificial ‘‘blocky’’

pattern superimposed on it.

Artifacts will affect the

performance of fingerprint
recognition.
 Results using WSQ compression
file size 45621 bytes
compression ratio: 12.9

Fine details are better

preserved.

No “blocky” artifacts.
 WSQ Algorithm

Target bit rate can set via a parameter, similar to

the "quality" parameter in JPEG.
 Compression ratio

• FBI’s target bit rate is around 0.75 bits per pixel (bpp)

• This corresponds to a compression ratio of

8/0.75=10.7

• Let’s compare WSQ with JPEG …

 Varying compression ratio (cont’d)
0.9 bpp compression
WSQ image, file size 47619 bytes, JPEG image, file size 49658 bytes,
compression ratio 12.4 compression ratio 11.9
 Varying compression ratio (cont’d)
0.75 bpp compression
WSQ image, file size 39270 bytes JPEG image, file size 40780 bytes,
compression ratio 15.0 compression ratio 14.5
 Varying compression ratio (cont’d)
0.6 bpp compression
WSQ image, file size 30987 bytes, JPEG image, file size 30081 bytes,
compression ratio 19.0 compression ratio 19.6
 JPEG Modes

• JPEG supports several different modes

– Sequential Mode
– Progressive Mode
– Hierarchical Mode
– Lossless Mode

• The default mode is “sequential”

– Image is encoded in a single scan (left-to-right, top-to-
bottom).
(see “Survey” paper)
 Progressive JPEG
• Image is encoded in multiple scans.
• Produce a quick, roughly decoded image when
transmission time is long.

Sequential

Progressive
 Progressive JPEG (cont’d)

• We’ll examine the following algorithms:

(1) Progressive spectral selection algorithm
(2) Progressive successive approximation algorithm
(3) Hybrid progressive algorithm
 Progressive JPEG (cont’d)

(1) Progressive spectral selection algorithm

– Group DCT coefficients into several spectral bands
– Send low-frequency DCT coefficients first
– Send higher-frequency DCT coefficients next
Example
 Progressive JPEG (cont’d)

(2) Progressive successive approximation algorithm

– Send all DCT coefficients but with lower precision.
– Refine DCT coefficients in later scans.
Example
 Example
after 0.9s after 1.6s

after 3.6s after 7.0s

 Progressive JPEG (cont’d)

(3) Combined progressive algorithm

– Combines spectral selection and successive approximation.
 Hierarchical JPEG

• Hierarchical mode encodes the image at different

resolutions.

• Image is transmitted in multiple passes with increased

resolution at each pass.
 Hierarchical JPEG (cont’d)

N/4 x N/4

N/2 x N/2

NxN