Scribd
Upload
8 views

Computer Vision

Uploaded by

TJ
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
8 views

Computer Vision

Uploaded by

TJ
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 37

Recap: projection

x = K R t  X

 x
u   s u0   r11 r12 r13 tx   
wv  =  0 v0  r21  y
 r22 r23 ty  
     z 
1   0 0 1  r31 r32 r33 t z   
1 
Relating multiple views

Figure Credit: Bundler: Structure from Motion (SfM) for Unordered Image Collections
Recap of Filtering

• Linear filtering is dot product at


each position 1 1 1

– Not a matrix multiplication


1 1 1

1 1 1
– Can smooth, sharpen, translate
(among many other uses)

• We can use the Fourier transform


to represent images in the
frequency domain.
– Filtering in the spatial domain is
multiplication in the frequency
domain.
Canvas Quiz Filtering Operator
1) _ = D * B A
Fill in the blanks: 2) A = _ * C
3) F = D * _
4) _ = D * D

E B
G

F
C

I
H D

Slide: Hoiem
Other signals
• We can also think of all kinds of other signals the same way

xkcd.com
Fourier analysis in images

Intensity Image

Fourier Image

http://sharp.bu.edu/~slehar/fourier/fourier.html#filtering
Fourier Transform
• Fourier transform stores the magnitude and phase at each frequency
– Magnitude encodes how much signal there is at a particular frequency
– Phase encodes spatial information (indirectly)
– For mathematical convenience, this is often notated in terms of real and complex numbers

I ( )
Amplitude: A =  R( ) + I ( )
2 2
Phase:  = tan −1

R( )
Salvador Dali invented Hybrid Images?

Salvador Dali
“Gala Contemplating the Mediterranean Sea,
which at 20 meters becomes the portrait
of Abraham Lincoln”, 1976
Fourier Bases
Teases away fast vs. slow changes in the image.

This change of basis is the Fourier Transform


Fourier Bases
This looks a lot like DCT in JPEG compression

8x8 image patch Patch representation after


projecting on to DCT bases
DCT bases
Man-made Scene
Can change spectrum, then reconstruct
Low and High Pass filtering
Computing the Fourier Transform

Continuous

Euler’s Formula
Discrete

k = -N/2..N/2

Fast Fourier Transform (FFT): NlogN


https://youtu.be/spUNpyF58BY?si=93x8YxT5n45OA3CD
The Convolution Theorem
• The Fourier transform of the convolution of two
functions is the product of their Fourier transforms

F[ g  h] = F[ g ] F[h]

• Convolution in spatial domain is equivalent to


multiplication in frequency domain!
−1
g * h = F [F[ g ] F[h]]
1 0 -1
Filtering in spatial domain 2 0 -2
1 0 -1

* =
Filtering in frequency domain

FFT

FFT

=
Inverse FFT

Slide: Hoiem
Filtering
Why does the Gaussian give a nice smooth
image, but the square filter give edgy artifacts?
Gaussian Box filter
Gaussian
Box Filter
Is convolution invertible?
• If convolution is just multiplication in the Fourier domain, isn’t
deconvolution just division?
• Sometimes, it clearly is invertible (e.g. a convolution with an
identity filter)
• In one case, it clearly isn’t invertible (e.g. convolution with an
all zero filter)
• What about for common filters like a Gaussian?
But you can’t invert multiplication by 0
• But it’s not quite zero, is it…
Let’s experiment on Novak
Convolution

* =

FFT FFT iFFT

.* =
Deconvolution?

iFFT FFT FFT

= ./
But under more realistic conditions
Random noise, .000001 magnitude

iFFT FFT FFT

= ./
But under more realistic conditions
Random noise, .0001 magnitude

iFFT FFT FFT

= ./
But under more realistic conditions
Random noise, .001 magnitude

iFFT FFT FFT

= ./
With a random filter…
Random noise, .001 magnitude

iFFT FFT FFT

= ./
Deconvolution is hard
• Active research area.
• Even if you know the filter (non-blind deconvolution), it is still
very hard and requires strong regularization.
• If you don’t know the filter (blind deconvolution) it is harder
still.
Blind Deconvolution Example

Edge-based Blur Kernel Estimation Using Patch Priors.


Libin Sun, Sunghyun Cho, Jue Wang, and James Hays.
IEEE International Conference on Computational Photography 2013.
Edge-based Blur Kernel Estimation Using Patch Priors.
Libin Sun, Sunghyun Cho, Jue Wang, and James Hays.
IEEE International Conference on Computational Photography 2013.

You might also like