General Image Transforms and Applications

Lecture 6, March 2nd, 2009 Lexing Xie

EE4830 Digital Image Processing http://www.ee.columbia.edu/~xlx/ee4830/
thanks to G&W website, Min Wu, Jelena Kovacevic and Martin Vetterli for slide materials

announcements
HW#2 due today HW#3 out Midterm next week, class time+location
Monday March 9th (4:10-6:40, Mudd 1127) Open-book
YES: text book(s), class notes, calculator NO: computer/cellphone/matlab/internet

5 analytical problems Coverage: lecture 1-6

intro, representation, color, enhancement, transforms and filtering (until DFT and DCT)

Additional instructor office hours

2-4pm Monday March 9th, Mudd 1312

Grading breakdown
HW-Midterm-Final: 30%-30%-40%

outline
Recap of DFT and DCT Unitary transforms KLT Other unitary transforms Multi-resolution and wavelets Applications

Readings for today and last week: G&W Chap 4, 7, Jain 5.1-5.11

recap: transform as basis expansion

inverse transform

recap: DFT and DCT basis

1D-DCT 1D-DFT

N=32

real(A)

imag(A)

recap: 2-D transforms

2D-DFT and 2D-DCT are separable transforms.

separable 2-D transforms

Symmetric 2D separable transforms can be expressed with the notations of its corresponding 1D transform.

We only need to discuss 1D transforms

Exercise
How do we decompose this picture?
DCT2

1 A2 = 2

1 1 1 1

=-1 =1

1 1 1 1

Four basis images for 2D DCT

=0 =1

0 0 0 1

DCT2

? ? ? ?

What if black=0, does the transform coefficients look similar?

two properties of DFT and DCT

Orthonormal (Eq 5.5 in Jain) : no two basis represent the same information in the image

Completeness (Eq 5.6 in Jain) : all information in the image are represented in the set of basis functions

minimized when

Unitary Transforms
A linear transform:

The Hermitian of matrix A is: This transform is called unitary when A is a unitary matrix, orthogonal when A is unitary and real.

Two properties implied by construction

Orthonormality Completeness

Exercise
Are these transform matrixes unitary/orthogonal?
2 3 1 2
1 2 1 2 1 2 1 2
cos sin sin cos

2 j

j 2

1 2

1 j j 1

Unitary/orthogonal checklist:
determinant equals 1, |A|=1 unit row/column vector orthogonal row/column vectors, AAH=I

properties of 1-D unitary transform

energy conservation

rotation invariance
the angles between vectors are preserved

unitary transform: rotate a vector in Rn, i.e., rotate the basis coordinates

observations about unitary transform

Energy Compaction Many common unitary transforms tend to pack a large fraction of signal energy into just a few transform coefficients De-correlation Highly correlated input elements quite uncorrelated output coefficients Use the covariance matrix to measure correlation

f: columns of image pixels

linear display scale: g

display scale: log(1+abs(g))

one question and two more observations

is there a transform with
best energy compaction maximum de-correlation is also unitary ?

transforms so far are data-independent

transform basis/filters do not depend on the signal being processed

optimal should be defined in a statistical sense so that the transform works well with many images
optimal for each signal is ill-defined

signal statistics should play an important role

review: correlation after a linear transform

x is a zero-mean random vector in
the covariance (autocorrelation) matrix of x
Rx(i,j) encodes the correlation between xi and xj Rx is a diagonal matrix iff. all N random variables in x are uncorrelated

apply a linear transform:

What is the correlation matrix for y ?

transform with maximum energy compaction

proof. maximum energy compaction

constraints:

matrix identity

a*u are the eigen vectors of Rx

Karhunen-Love Transform (KLT)

a unitary transform with the basis vectors in A being the orthonormalized eigenvectors of Rx

assume real input, write AT instead of AH denote the inverse transform matrix as A, AAT=I Rx is symmetric for real input, Hermitian for complex input i.e. RxT=Rx, RxH = Rx Rx nonnegative definite, i.e. has real non-negative eigen values Attributions
Kari Karhunen 1947, Michel Love 1948 a.k.a Hotelling transform (Harold Hotelling, discrete formulation 1933) a.k.a. Principle Component Analysis (PCA, estimate Rx from samples)

Properties of K-L Transform

Decorrelation by construction

note: other matrices (unitary or nonunitary) may also de-correlate the transformed sequence [Jains example 5.5 and 5.7]

Minimizing MSE under basis restriction

Basis restriction: Keep only a subset of m transform coefficients and then perform inverse transform (1 m N) Keep the coefficients w.r.t. the eigenvectors of the first m largest eigenvalues

discussions about KLT

The good
Minimum MSE for a shortened version De-correlating the transform coefficients

The ugly
Data dependent
Need a good estimate of the second-order statistics Increased computation complexity
data: linear transform: fast transform: estimate Rx: compute eig Rx:

Is there a data-independent transform with similar performance?

DFT is the optimal transform when

The signal x is periodic The autocorrelation matrix Rx is circulant

The eigen vectors of Rx are Fourier basis

energy compaction properties of DCT

DCT is close to KLT when ...
x is first-order stationary Markov

Rx and 2 Rx-1 have the same eigen vectors 2 Rx-1 ~ Qc when is close to 1 DCT basis vectors are eigenvectors of a symmetric tri-diagonal matrix Qc

verify with trigonometric identity: cos(a+b)+cos(a-b)=2cos(a)cos(b)

DCT energy compaction

DCT is close to KLT for
highly-correlated first-order stationary Markov source

DCT is a good replacement for KLT

Close to optimal for highly correlated data Not depend on specific data Fast algorithm available

DCT/KLT example for vectors

x: columns of image pixels * = 0.8786
fraction of coefficient values in the diagonal

0.0136

0.1055
transform basis

0.1185

1.0000

display scale: log(1+abs(g)), zero-mean

KL transform for images

autocorrelation function 1D 2D

KL basis images are the orthonormalized eigen-functions of R

rewrite images into vector forms (N2x1)

solve the eigen problem for N2xN2 matrix ~ O(N6)

or, make Rx separable

perform separate KLT on the rows and columns transform complexity O(N3)

KLT on hand-written digits

1100 vectors of size 256x1

1100 digits 6 16x16 pixels

The Desirables for Image Transforms

Theory
Inverse transform available Energy conservation (Parsevell) Good for compacting energy Orthonormal, complete basis (sort of) shift- and rotation invariant Transform basis signal-independent

DFT DCT

KLT

?
?

Implementation
Real-valued Separable Fast to compute w. butterfly-like structure Same implementation for forward and inverse transform
x x x x

Walsh-Hadamard Transform

slant transform

Nassiri et. al, Texture Feature Extraction using Slant-Hadamard Transform

energy compaction comparison

implementation note: block transform

similar to STFT (short-time Fourier transform)
partition a NxN image into mxn sub-images save computation: O(N) instead of O(NlogN) lose long-range correlation

8x8 DCT coefficients

applications of transforms
enhancement (non-universal) compression feature extraction and representation pattern recognition, e.g., eigen faces dimensionality reduction
analyze the principal (dominating) components

Image Compression

Measure compression quality with signal distortion:

where P is average power and A is RMS amplitude.

Gabor filters
Gaussian windowed Fourier Transform
Make convolution kernels from product of Fourier basis images and Gaussians

Odd (sin) Even (cos)

Frequency

Example: Filter Responses

Filter bank

Input image

from Forsyth & Ponce

outline
Recap of DFT and DCT Unitary transforms KLT Other unitary transforms Multi-resolution and wavelets Applications

sampling (dirac)

FT

STFT

FT does not capture discontinuities well

one step forward from dirac

Split the frequency in half means we can downsample by 2 to reconstruct upsample by 2. Filter to remove unwanted parts of the images and add Basic building block: Two-channel filter bank

analysis

processing

synthesis

h
x

h
x

orthogonal filter banks

1.

Start from the reconstructed signal

Read off the basis functions

orthogonal filter banks

2.

We want the expansion to be orthonormal

The output of the analysis bank is

3.

Then
The rows of T are the basis functions The rows of T are the reversed versions of the filters

The analysis filters are

orthogonal filter banks

4.

Since is unitary, basis functions are orthonormal

5.

Final filter bank

orthogonal filter banks: Haar basis

DWT
Iterate only on the lowpass channel
f

wavelet packet
Iterate on both the low pass and (selected) high-pass channels

wavelet packet
First stage: full decomposition

wavelet packet
Second stage: pruning
Cost(parent) < Cost(children)

wavelet packet: why it works

One of the grand challenges in signal analysis and processing is in understanding blob-like structures of the energy distribution in the time-frequency space, and designing a representation to reflect those.
Dirac basis

FT

STFT

t f WT f WP

are we solving x=x?

sort of: find matrices such that after finding those
Decomposition Reconstruction

in a nutshell
if is if is = if is frame if is square and nonsingular, is a basis and is its dual basis unitary, that is, * = I, is an orthonormal basis and rectangular and full rank, is a frame and is its dual =

rectangular and * = I , is a tight frame and

overview of multi-resolution techniques

applications of wavelets
enhancement and denoising compression and MR approximation fingerprint representation with wavelet packets bio-medical image classification subdivision surfaces Geris Game, A Bugs Life, Toy Story 2

fingerprint feature extraction

MR system
Introduces adaptivity Template matching performed on different space-frequency regions Builds a different decomposition for each class

fingerprint identification results

100 90 Identification Error Rate (%) 80 70 60 50 40 30 20 10 0
1 SCF WDCF 0 0 2 0 0 3 9.78 0 4 3.26 0 5 6 7 8 9 10 9.78 0 11 0 0 12 13 14 15 16 17 18 19 0 0 20 11.96 0 4.35 35.90 33.70 89.96 6.52 0 5.43 0 0 0 3.26 66.30 15.22 8.70 21.74 33.70 14.13 0 15.22 0 0 13.04 0 0

Standard Correlation Filters Wavelet Correlation Filters

SCF Average IER = 18.41%

WDCF Average IER = 1.68%

NIST 24 fingerprint database 10 people (5 male & 5 female), 2 fingers 20 classes, 100 images/class

references for multiresolution

Light reading
Wavelets: Seeing the Forest -- and the Trees, D. Mackenzie, Beyond Discovery, December 2001.

Overviews

Books
Wavelets and Subband Coding, M. Vetterli and J. Kovacevic, Prentice Hall, 1995. A Wavelet Tour of Signal Processing, S. Mallat, Academic Press, 1999. Ten Lectures on Wavelets, I. Daubechies, SIAM, 1992. Wavelets and Filter Banks, G. Strang and T. Nguyen, Wells. Cambr. Press, 1996.

ELEN E6860 Advanced Digital Signal Processing

summary
unitary transforms
theory revisited the quest for optimal transform example transforms DFT, DCT, KLT, Hadamard, Slant, Haar,

multi-resolution analysis and wavelets applications

compression feature extraction and representation image matching (digits, faces, fingerprints)

10 yrs

1 yr