Presenter:

J in-Zuo Liu
Research Advisor:
J ian-J iun Ding , Ph. D.
Digital Image and Signal Processing Lab
Graduate Institute of Communication Engineering
National Taiwan University



Image Compression
1
Outlines
Introduction to Image compression
JPEG Standard
JPEG2000 Standard
Shape-Adaptive Image Compression
Modified JPEG Image Compression
Conclusions
Reference



2
Image Storage System

Object
R-G-B
coordinate
Transform to
Y-Cb-Cr
coordinate
Downsample
Chrominance
Encoder
Decoder
R-G-B
coordinate
Transform to
R-G-B
coordinate
Upsample
Chrominance
Monitor
C
Camera
C
V
HDD
Performance
RMSE
PSNR
3

Transform function:






Y: the luminance represents the brightness
Cb: the difference between the gray and blue
Cr: the difference between the gray and red
0.299 0.587 0.114 0
0.169 0.334 0.500 128
0.500 0.419 0.081 128
Y R
Cb G
Cr B
( ( ( (
( ( ( (
= +
( ( ( (
( ( ( (

4
Downsampling formats of YCbCr
Y
Cb
Cr
W
W
W
H
H
H
Y
W
H Y
W
H
Cb
W/2
H
Cr
W/2
H
Cb
W/2
H/2
Cr
W/2
H/2
(a) 4 : 4 : 4 (b) 4 : 2 : 2 (c) 4 : 2 : 0
Y
W
H
C
b
H
Cr
W/4
H
(d) 4 : 1 : 1
W/4
5
Performance measures

n
1
the data quantity of original
image
n
2
the data quantity of the
generated bitstream.



W
the width
H
the height of the

original image


1
2
=
n
CR
n
| |
1 1
2
0 0
( , ) '( , )
W H
x y
f x y f x y
RMSE
WH

= =

=

10
255
20log PSNR
MSE
=
6
Outlines
Introduction to Image compression
JPEG Standard
JPEG2000 Standard
Shape-Adaptive Image Compression
Modified JPEG Image Compression
Conclusions
Reference



7
JPEG flowchart

Image
R
G
B
YCbCr Color
Transform
Chrominance
Downsampling
(4:2:2 or 4:2:0)
8 8
FDCT
Quantizer
Quantization
Table
Zigzag &
Run Length
Coding
Differential
Coding
Huffman
Encoding
Huffman
Encoding
Bit-stream
8
Why we apply DCT?
Reduce the correlation between the neighboring
pixels in the image
coordinate rotation

the f2th pixel value Y



X
the f1th pixel value



f1-f2 = 3 pixels in horizontal
9
Covariance Matrix

Step1: Image partition
Step2: Re-aligned the pixels of a 2-D block into a
1-D vector






( ) ( ) ( ) ( )
( ) ( ) ( ) ( )
1 1 1
1
m m m m
N
xx
m m m m
N N N
E x x E x x
C
E x x E x x
(
( (

(
(
=
(
( (
(


10
Karhunen-Loeve Transform (KLT)
Coordinate rotation


Normal orthogonal transformation





V = [ v1 v2 vN ]
vi the eigenvector of the corrosponding
eigenvalue i of Cxx ( i 1 N )

( ) ( )
m m
t
= y V x
( ) ( )
( )
( ) ( )
( )
t t
m m m m
t t t t
yy xx
C E E C
( (
= = =
( (

V x V x V x x V V V
11
DCT V.S KLT
KLT is the Optimal Orthogonal Transform with minimal MSE
but is difficult to implement

DCT is the limit situation of KLT

DCT advantages:

1. Eliminate the dependence on image data
2. Obtain the general transformation for every
image
3. Reduce the correlation between pixels just
like KLT
4. Smaller computation time
5. Real numbers



12
Discrete Cosine Transform (DCT)
Forward 2-D Discrete Cosine Transform





Inverse 2-D Discrete Cosine Transform



f(x,y) : the element in spatial domain
F(u,v) : the DCT coefficient in the frequency domain

7 7
0 0
1 (2 1) (2 1)
( , ) ( ) ( ) ( , ) cos cos
4 16 16
for 0,..., 7 and 0,..., 7
1/ 2 for 0
where ( )
1 otherwise
x y
x u y v
F u v C u C v f x y
u v
k
C k
t t
= =
+ +
( (
=
( (

= =

=
=

1 1
0 0
2 (2 1) (2 1)
( , ) ( ) ( ) ( , ) cos cos
2 2
for 0,..., 1 and 0,..., 1
N N
u v
x u y v
f x y C u C v F u v
M N
MN
x M y N
t t

= =
+ +
( (
=
( (

= =

13
Discrete Cosine Transform (DCT)
88 DCT
14
JPEG Quantization
Qantization:


Qantization table
( , )
( , )
( , )
Quantization
F u v
F u v round
Q u v
| |
=
|
\ .
15
DPCM for DC Components
large correlation still exists between the DC
components in the neighboring macroblocks
DCi-1 DCi
Blocki-1 Blocki

Diff
i
= DC
i
- DC
i-1
16
Grouping method for DC component


Values Bits for the value

group
0 0
-1,1 0,1 1
-3,-2,2,3 00,01,10,11 2
-7,-6,-5,-4,4,5,6,7 000,001,010,011,100,101,110,111 3
-15,...,-8,8,...,15 0000,...,0111,1000,...,1111 4
-31,...,-16,16,...31 00000,...,01111,10000,...,11111 5
-63,...,-32,32,...63 000000,...,011111,100000,...,111111 6
-127,...,-64,64,...,127 0000000,...,0111111,1000000,...,1111111 7
-255,..,-128,128,..,255 ... 8
-511,..,-256,256,..,511 ... 9
-1023,..,-512,512,..,1023 ... 10
-2047,...,-1024,1024,...,2047 ... 11
17
Grouping method for DC component







Example: diff=17
(17)10 = (10001)2
group 5 codeword: (110)2
code: (11010001)2

18
Zigzag Scanning of the AC
Coefficients
19
Run Length Coding of the AC
Coefficients
The RLC step replaces the quantized values by



Example:
the zig-zag scaned 63 AC coefficients:


Perform RLC :

( , ) RUNLENGTH VALUE
the number of
zeros
the nonzero
coefficients
| |
57, 45, 0, 0, 0, 0, 23, 0, 30, 16, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, ..., 0
( ) ( ) ( ) ( ) ( ) ( )
0, 57 0, 45 4, 23 1, 30 0, 16 2,1 EOB (

20
The Run/Size Huffman table for the
luminance AC coefficients








Run/Size code length code word
0/0 (EOB) 4 1010
15/0 (ZRL) 11 11111111001
0/1 2 00
...
0/6 7 1111000
...
0/10 16 1111111110000011
1/1 4 1100
1/2 5 11011
...
1/10 16 1111111110001000
2/1 5 11100
...
4/5 16 1111111110011000
...
15/10 16 1111111111111110
21
Outlines
Introduction to Image compression
JPEG Standard
JPEG2000 Standard
Shape-Adaptive Image Compression
Modified JPEG Image Compression
Conclusions
Reference



22
The JPEG 2000 Standard

JPEG2000 fundamental building blocks
Image
R
G
B
Forward
Component
Transform
2D DWT Quantization EBCOT
Context
Modeling
Arithmetic
Coding
Rate-
Distortion
Control
Tier-2
Tier-1
J PEG 2000
Bit-stream
23
Discrete Wavelet Transform
The analysis filter bank of the 2-D DWT

2
2
( 1, , ) W j m n

+
( ) h n

( ) h n

2
2
( ) h m

2
2
( ) h m

( , , )
D
W j m n

( , , )
V
W j m n

( , , )
H
W j m n

( , , ) W j m n

Columns
Rows
( ) h m

( ) h m

24
Wavelet Transforms in Two Dimension
Two-scale of 2-D decomposition
( , , )
D
W j m n

( , , )
V
W j m n

( , , )
H
W j m n

( , , ) W j m n

( 1, , ) W j m n

+
LL2
LH2
LH1 HH1
HL1
HL2
HH2
25
Discrete Wavelet Transform








One-scale of 2-D DWT
26
Outlines
Introduction to Image compression
JPEG Standard
JPEG2000 Standard
Shape-Adaptive Image Compression
Modified JPEG Image Compression
Conclusions
Reference



27
Shape-Adaptive Image Compression

Block-based transformation disadvantages:

1. block effect

2. no take advantage of the local
characteristics in an image segment
28
Shape-Adaptive Image Compression

Algorithm structure
Image
Segmentation
Boundary
Transform
Coding
Arbitrary Shape
Transform
Coding
Quantization
And
Entropy Coding
Quantization
And
Entropy Coding
Bit-stream
Boundary
Interal texture
Boundary Descriptor
Coefficients of Transform Bases
29
Shape-Adaptive
Transformation(1)

Padding Algorithm
Padding zeros into the pixel positions out of the
image segment
0 0 0 0 75 96 0 0
105 98 99 101 73 85 66 60
100 97 0
89 94 87 64 55
0 0 84
0 0 0
0 0 93
0 0 0 105 104 0 0 0
0 0 0
94 90 81 71 66
86 86 81 72 0
86 94 81 70 0
98 97 78 0 0
30
Shape-Adaptive Transformation(2)
Arbitrarily-Shaped DCT Bases



For and , where



W: the width of the image segment
H: the height of the image segment
1 1
0 0
2 ( ) ( ) (2 1) (2 1)
( , ) ( , ) cos cos
2 2
*
W H
x y
C u C v x u y v
F u v f x y
W H
H W
t t

= =
+ +
( (
=
( (

0,..., 1 u W = 0,..., 1 v H =
1/ 2 for 0
( )
1 otherwise
k
C k
=
=

31
Shape-Adaptive Transformation(2)
Arbitrarily-Shaped DCT Bases







0 0 0 0 1 1 0 0
1 1 1 1 1 1 1 1
1 1 0
1 1 1 1 1
0 0 1
0 0 0
0 0 1
0 0 0 1 1 0 0 0
0 0 0
1 1 1 1 1
1 1 1 1 0
1 1 1 1 0
1 1 1 0 0
0 1 2 3 4 5 6 7
0
1
2
3
4
5
6
7
The shape matrix
The 8x8 DCT bases with the shape
32
Gram-Schmidt algorithm
The 37 arbitrarily-shape orthogonal DCT bases

1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30
31 32 33 34 35 36 37
33
Shape-Adaptive Transformation(3)
Shape-Adaptive DCT Algorithm ( SADCT )

x
y
x x
y' u
x
u
x' v
u u
D
C
T
-
1
D
C
T
-
2
D
C
T
-
3
D
C
T
-
4
D
C
T
-
3
D
C
T
-
6
DCT-6
DCT-5
DCT-4
DCT-2
DCT-1
DCT-1
(a) (b)
(c)
(d) (e)
(f)
DC values
DC coefficients
34
Shape-Adaptive DCT Algorithm
( SADCT )
The variable length (N-point) 1-D DCT transform
matrix DCT-N


: the pth DCT basis vector


Transform function:






(2 / ) DCT-N
j j
c N x =
0
1
DCT-N( , ) cos , , 0 1
2
p k c p k k p N
N
t (
| |
= + =
| (
\ .

1/ 2 for 0
( )
1 otherwise
p
C p
=
=

p

35
Morphological Erosion
Input Image
Image
Segmentation
Morphological
Operation (Erosion)
Shape
Adaptive DCT
Quantization Entropy Coding
Output
Bitstream
36



Morphological Erosion


Contour sub-region
Interior sub-region
The overall
object
37
Morphological Erosion

Algorithm structure


Interior sub-regions
Contour sub-regions
Shape-
adaptive
DCT
Shape-
adaptive
DCT
Segmentation
+
boundaries extraction
38
Shape-Adaptive Image Compression


1010101011011111
Image segments
100111101010
101010111111111001
111000111000101011111
Quantizing & encoding
EOB
EOB
EOB
DCT coefficients
boundary
encoding
bit stream of
boundaries
100111101010 1010101111111110
111000111000101011111
EOB
EOB EOB 01
M
1

M
2

M
3

S.A.
DCT
Bit-stream of image
segments
combine
39
Simulation Results

0.4 0.5 0.6 0.7 0.8 0.9 1 1.1
38
38.5
39
39.5
40
40.5
41
41.5
42
42.5
43
Bitrate(10
4
) [Bits]
P
S
N
R
















[
d
B
]


SADCT
SADCT with erosion
SADCT SADCT with erosion
SADCT SADCT with erosion operation
40
Outlines
Introduction to Image compression
JPEG Standard
JPEG2000 Standard
Shape-Adaptive Image Compression
Modified JPEG Image Compression
Conclusions
Reference



41
Modified JPEG Image
Compression
2-D Orthogonal DCT Expansion in
Triangular and Trapezoid Regions
p
All 8X8 rectangular blocks
Triangular, trapezoid or rectangular
blocks
(b) (a)
42
Trapezoid Definition

Define the trapezoid :
(M-1)
th
row
(M-2)
th
row
1
st
row
0
th
row
.
.
.
.
.
.
( ) ( )
1 is a constant. K m K M m +
43
Trapezoid Definition
Shearing a region that satisfies into the trapezoid
region whose first pixels in each row are aligned
at the same column.




A triangular region can be viewed as a special
case of the trapezoid region where

Shearing
(b) (a)
(M-1)
th
row
1
st
row
0
th
row
.
.
.
.
44
Complete and Orthogonal DCT
Basis in the Trapezoid Region


m = M-1
m = M-2
m = 1
m = 0
.
.
.
.
.
.
n = 0 1 2
Region A
Region B
Rotation by 180
Region A
Region B
Rectangular Region
(a)
(b)
45
Complete and Orthogonal DCT
Basis in the Trapezoid Region



2 4 6 8 10
2
4
2 4 6 8 10
2
4
2 4 6 8 10
2
4
2 4 6 8 10
2
4

2 4 6 8 10
2
4
2 4 6 8 10
2
4
2 4 6 8 10
2
4
2 4 6 8 10
2
4

2 4 6 8 10
2
4
2 4 6 8 10
2
4
2 4 6 8 10
2
4
2 4 6 8 10
2
4

2 4 6 8 10
2
4
2 4 6 8 10
2
4
2 4 6 8 10
2
4
2 4 6 8 10
2
4
(b)
(a)
46
Finding an approximate trapezoid
region in an arbitrary shape

(a) (b)
approximate
(a) (b)
47

Modified JPEG Image Compression
Divide Images into three regions:
Trapezoid and triangular
regions
Traditional 8X8 image
blocks
8X8 SADCT image blocks
48
Simulation Results

50 100
50
100

50 100
50
100

(a) JPEG 692 Bytes (b) Proposed scheme 165 Bytes
49
Simulation Results
50
Reference
[1] R. C. Gonzalea and R. E. Woods, "Digital Image
Processing", 2nd Ed., Prentice Hall, 2004.
[2] Liu Chien-Chih, Hang Hsueh-Ming, "Acceleration and
Implementation of JPEG 2000 Encoder on TI DSP
platform" Image Processing, 2007. ICIP 2007. IEEE
International Conference on, Vo1. 3, pp. III-329-339,
2005.
[3] ISO/IEC 15444-1:2000(E), "Information technology-
JPEG 2000 image coding system-Part 1: Core coding
system", 2000.
[4] Jian-Jiun Ding and Jiun-De Huang, "Image
Compression by Segmentation and Boundary
Description", Masters Thesis, National Taiwan University,
Taipei, 2007.
[5] Jian-Jiun Ding and Tzu-Heng Lee, "Shape-Adaptive
Image Compression", Masters Thesis, National Taiwan
University, Taipei, 2008.
[6] G. K. Wallace, "The JPEG Still Picture Compression
Standard", Communications of the ACM, Vol. 34, Issue 4,
pp.30-44, 1991.
[7] JPEG 2000
() IC 2003.8.
[8]
2004.

51
Thank you for
listening ~
52