1

IMAGE INTERPOLATION
Francesca Pizzorni Ferrarese

Image Interpolation
2

Introduction
What

Interpolation Techniques
1D

is image interpolation? (D-A conversion) Why do we need it? zero-order, first-order, third-order 2D = two sequential 1D (divide-and-conquer) Directional(Adaptive) interpolation*
Digital

Interpolation Applications

zooming (resolution enhancement) Image inpainting (error concealment) Geometric transformations (where your imagination can fly)

Introduction
3

What is image interpolation?

image f(x,y) tells us the intensity values at the integral lattice locations, i.e., when x and y are both integers Image interpolation refers to the guess of intensity values at missing locations, i.e., x and y can be arbitrary Note that it is just a guess (Note that all sensors have finite sampling distance)

A Sentimental Comment
4

Havent we just learned from discrete sampling (A-D conversion)? Yes, image interpolation is about D-A conversion Recall the gap between biological vision and artificial vision systems

Digital:

camera + computer Analog: retina + brain

Engineering Motivations
5

Why do we need image interpolation?

want BIG images want GOOD images want COOL images

When

we see a video clip on a PC, we like to see it in the full screen mode some block of an image gets damaged during the transmission, we want to repair it images digitally can render fancy artistic effects as we often see in movies

We
If

We

Manipulate

Scenario I: Resolution Enhancement

Low-Res. High-Res.

Scenario II: Image Inpainting

Non-damaged

Damaged

Scenario III: Image Warping

Image Interpolation
9

Introduction

Interpolation Techniques

What is image interpolation? Why do we need it? 1D linear interpolation (elementary algebra) 2D = 2 sequential 1D (divide-and-conquer) Directional(adaptive) interpolation* Digital zooming (resolution enhancement) Image inpainting (error concealment) Geometric transformations

Interpolation Applications

Upsampling
This image is too small for this screen: How can we make it 10 9mes as big? Simplest approach: repeat each row and column 10 9mes ( Nearest neighbor interpola9on )

Image interpola9on
d = 1 in this example
1 2 3 4 5

Recall how a digital image is formed

It is a discrete point-sampling of a con9nuous func9on If we could somehow reconstruct the original func9on, any new image could be generated, at any resolu9on and scale
Adapted from: S. Seitz

Image interpola9on
d = 1 in this example
1 2 3 4 5

Recall how a digital image is formed

It is a discrete point-sampling of a con9nuous func9on If we could somehow reconstruct the original func9on, any new image could be generated, at any resolu9on and scale
Adapted from: S. Seitz

Image interpola9on
d = 1 in this example
1 2 2.5 3 4 5

What if we don t know ?

Image interpolation
Ideal reconstruc9on

Nearest-neighbor interpola9on

Linear interpola9on

Gaussian reconstruc9on
Source: B. Curless

Ideal reconstruction
15

Ideal reconstruction
16

Ideal reconstruction
17

Image interpolation
Original image: x 10

Nearest-neighbor interpola9on

Bilinear interpola9on

Bicubic interpola9on

1D Zero-order (Replication)
19

f(n) n

f(x) x

1D First-order Interpolation (Linear)

f(n) n

f(x) x

Linear Interpolation Formula

Heuristic: the closer to a pixel, the higher weight is assigned Principle: line fitting to polynomial fitting (analytical formula) f(n) f(n+a) f(n+1)

1-a

f(n+a)=(1-a)f(n)+af(n+1), 0<a<1 Note: when a=0.5, we simply have the average of two

Numerical Examples
22

f(n)=[0,120,180,120,0] Interpolate at 1/2-pixel f(x)=[0,60,120,150,180,150,120,60,0], x=n/2 Interpolate at 1/3-pixel f(x)=[0,20,40,60,80,100,120,130,140,150,160,170,180,], x=n/6

1D Third-order Interpolation (Cubic)*

f(n) n f(x) x Cubic spline fitting

From 1D to 2D
24

Engineers wisdom: divide and conquer 2D interpolation can be decomposed into two sequential 1D interpolations. The ordering does not matter (row-column = column-row) Such separable implementation is not optimal but enjoys low computational complexity If you dont know how to solve a problem, there must be a related but easier problem you know how to solve. See if you can reduce the problem to the easier one. - rephrased from G. Polyas How to Solve It

25

Graphical Interpretation of Interpolation at Half-pel

row

column

f(m,n)

g(m,n)

Numerical Examples
26

a c

b d first-order

zero-order a a c c a a c c b b d d b b d d

a (a+c)/2 c

(a+b)/2 (a+b+c+d)/4 (c+d)/2

b (b+d)/2 d

Numerical Examples (Cont)

Col n row m X(m,n) a b Y 1-a

Col n+1 X(m,n+1)

1-b row m+1 X(m+1,n) X(m+1,n+1)

Q: what is the interpolated value at Y? Ans.: (1-a)(1-b)X(m,n)+(1-a)bX(m+1,n) +a(1-b)X(m,n+1)+abX(m+1,n+1)

Bicubic Interpolation*
28

http://en.wikipedia.org/wiki/Bicubic_interpolation

Limitation with bilinear/bicubic

Edge blurring Jagged artifacts

Edge blurring

Jagged artifacts

Z X X Z

Edge-Sensitive Interpolation
30

Step 1: interpolate the missing pixels along the diagonal a x c d b Since |a-c|=|b-d| x has equal probability of being black or white

black or white?

Step 2: interpolate the other half missing pixels a d c x b Since |a-c|>|b-d| x=(b+d)/2=black

Image Interpolation
31

Introduction Interpolation Techniques

Interpolation Applications
Digital zooming (resolution enhancement) Image inpainting (error concealment) Geometric transformations (where your imagination can fly)

Pixel Replication
32

low-resolution image (100100) high-resolution image (400400)

Bilinear Interpolation
33

low-resolution image (100100)

high-resolution image (400400)

Bicubic Interpolation
34

low-resolution image (100100)

high-resolution image (400400)

35

Edge-Directed Interpolation (Li&Orchard2000)

low-resolution image (100100)

high-resolution image (400400)

Image Demosaicing (Color-Filter-Array Interpolation)

Bayer Pattern

Image Example
37

Ad-hoc CFA Interpolation

Advanced CFA Interpolation

Error Concealment*
38

damaged

interpolated

Image Inpainting*
39

Image Mosaicing*
40

Geometric Transformation
41

MATLAB functions: griddata, interp2, maketform, imtransform

Basic Principle
42

(x,y) (x,y) is a geometric transformation We are given pixel values at (x,y) and want to interpolate the unknown values at (x,y) Usually (x,y) are not integers and therefore we can use linear interpolation to guess their values

MATLAB implementation: z=interp2(x,y,z,x,y,method);

Rotation
43

x x

x' cos y ' = sin

sin x y cos

MATLAB Example
44

z=imread('cameraman.tif'); % original coordinates [x,y]=meshgrid(1:256,1:256); % new coordinates a=2; for i=1:256;for j=1:256; x1(i,j)=a*x(i,j); y1(i,j=y(i,j)/a; end;end % Do the interpolation z1=interp2(x,y,z,x1,y1,'cubic');

Rotation Example
45

=3o

Scale
46

a=1/2

x' a 0 x y ' = 0 1 / a y

Affine Transform
47

square

parallelogram

x' a11 a12 x d x + y ' = a 21 a22 y d y

Affine Transform Example

x' .5 1 x 0 y ' = .5 2 y + 1

Shear
49

square

parallelogram
x' 1 0 x d x y ' = s 1 y + d y

Shear Example
50

x' 1 0 x 0 y ' = .5 1 y + 1

Projective Transform
51

B A D B A C

C D

square

x '=

a1 x + a2 y + a3 a7 x + a8 y + 1

quadrilateral

a4 x + a5 y + a6 y' = a7 x + a8 y + 1

Projective Transform Example

[ 0 0; 1 0; 1 1; 0 1]

[-4 2; -8 -3; -3 -5; 6 3]

Polar Transform
53

y = tan x
1

r = x2 + y 2

Iris Image Unwrapping

Use Your Imagination

r -> sqrt(r)

http://astronomy.swin.edu.au/~pbourke/projection/imagewarp/

Free Form Deformation

Seung-Yong Lee et al., Image Metamorphosis Using Snakes and Free-Form Deformations,SIGGRAPH1985, Pages 439-448

Application into Image Metamorphosis

58

Summary of Image Interpolation