Imaging, Robotics, and Intelligent Systems

IRIS
Point Spread Function (PSF) Measurement for Cell Phone Camera
with a High Resolution PSF of the Imaging Lens and a Sub-pixel
Digital Algorithm

Ingrid Narvaez
Spring 2013

Dr. Mongi Abidi

 Abstract
Imaging, Robotics, and Intelligent Systems

IRIS
• An accurate measurement of the PSF for the
extended depth of field (EDoF) cell phone
camera with CMOS sensor is of great
importance for image processing and image
restoration.
• The overall system including the imaging
subsystem and sampling subsystem is a shift-
variant system with respect to the sample-
scene phase parameter within the sub-pixel
range.
• This paper talks about the sub-pixel digital
algorithm to estimate the overall camera PSF.
Slide 2
 Abstract
Imaging, Robotics, and Intelligent Systems

IRIS
• The sub-pixel digital algorithm
– Find the average of the shifted high resolution
PSFs of the lens over one active pixel area with the
assumption of uniform random distribution of point
source location within the active pixel area.
– The averaged high resolution PSF is down sampled
to the Bayer plane to obtain the shift invariant
overall system PSF.
– Finally this shift-invariant PSF is applied for image
restoration of blurred images captured with an
extended depth of field camera.

Slide 3
 Introduction
Imaging, Robotics, and Intelligent Systems

IRIS
• Because almost all available algorithms for
image reconstruction require an accurate
measurement of the system PSF, there is a
need to measure it accurately.
• In this method, a cell phone camera was
used, and the system for image acquisition is
as follows:

Slide 4
 Introduction
Imaging, Robotics, and Intelligent Systems

IRIS
• This paper focuses its attention in the
calculation of the PSF of the two subsystems
before the imaging pipeline.
• The input scene 𝑂(𝑥, 𝑦; 𝑥0 , 𝑦0 ), and the intensity
PSF of the lens aperture is ℎ(𝑥, 𝑦; 𝑥0 , 𝑦0 ) where
(𝑥, 𝑦) represents an image coordinate and
(𝑥0 , 𝑦0 ) is an object coordinate.
• The image 𝐼 is expressed mathematically as
∞
𝐼 𝑥, 𝑦; 𝑥0 , 𝑦0 = න 𝑂 𝑥0 , 𝑦0 ℎ(𝑥, 𝑦; 𝑥0 , 𝑦0 )𝑑𝑥0 𝑑𝑦0
−∞

Slide 5
 Introduction
Imaging, Robotics, and Intelligent Systems

IRIS
• The image 𝐼 is sampled by the a CMOS sensor
whose pixel size is ∆. The sampled image 𝐼𝑠 is
related to image I mathematically as:
1 𝑥 𝑦
𝐼𝑠 𝑥, 𝑦, 𝑥0 , 𝑦0 = 𝐼 𝑥, 𝑦, 𝑥0 , 𝑦0 ⊗ 𝐴(𝑥, 𝑦) 2 𝑐𝑜𝑚𝑏 ,
∆ ∆ ∆
• Where ⊗operator stands for convolution.
• 𝐴(𝑥, 𝑦) is the function which determines the
response over the pixel area.
• And the fill factor of the sensor
1 𝑥 𝑦
2
𝑐𝑜𝑚𝑏 , = ෍ ෍ 𝛿(𝑥 − Δ𝑚, 𝑦 − Δ𝑛)
∆ ∆ ∆
𝑛 𝑚
Slide 6
 Introduction
Imaging, Robotics, and Intelligent Systems

IRIS
• The overall system intensity PSF of the optics and
CMOS sensor ℎ𝑠 is expressed as follows
1 𝑥 𝑦
ℎ𝑠 𝑥, 𝑦; 𝑥0 , 𝑦0 = ℎ 𝑥, 𝑦; 𝑥0 , 𝑦0 ⊗ 𝐴(𝑥, 𝑦) 2 𝑐𝑜𝑚𝑏 ,
∆ ∆ ∆
• This paper deals with a point source of the PSF,
because it can be rotationally non-symmetric.
• The PSF shape varies a lot even tough the object is
located at the center of the camera.
• The PSF is dependent on the relative location (𝑥𝑟 , 𝑦𝑟 ) of
the center of the point source to the sampling grid. This
relative location is referred to as sample-scene
parameter.
• Due to insufficient sampling of the PSF, the system is
not shift-invariant even for a small field of view.
Slide 7
The Measurement of the Intensity Imaging, Robotics, and Intelligent Systems

PSF for Cell Phone Cameras IRIS

• Instead of measuring the overall system PSF which is
dependent on the relative location of the point source
with the sampling grid, the authors measured the
High Resolution PSF of the lens only.

• After obtaining the High Resolution PSF, it was down-

sampled to the sensor or Bayer plane with the
proposed sub-pixel algorithm.

• The High Resolution PSF is sampled with a sampling

△
grid of . Where 𝑚 is chosen to be large enough so
𝑚
that the measured digital High Resolution PSF could
accurately represent the actual PSF of the lens.
Slide 8
The Measurement of the Intensity Imaging, Robotics, and Intelligent Systems

PSF for Cell Phone Cameras IRIS

• The High Resolution PSF is digitally shifted with sub-pixel
△
step of .
10
• Because the sensor has periodic structure, the authors set
the range of shift with one pixel of the sensor.
• The center of the shifted High Resolution PSF is at the
sample scene parameter or (𝑥𝑟 , 𝑦𝑟 ).
• The averaged High Resolution PSF is obtained by
averaging all shifted high resolution PSFs ℎ(𝑥 − 𝑥𝑟 , 𝑦 −

Slide 9
The Measurement of the Intensity Imaging, Robotics, and Intelligent Systems

PSF for Cell Phone Cameras IRIS

• High resolution PSF of the lens
– The overall PSF of a cell phone camera is hard to
measure directly a faster and less complicated
approach is to align the system correctly.
– Two ways of measuring the high resolution PSF of the
lens-only were considered.

1. A collimated laser source that allows both the on-

axis and off-axis PSFs to be available.
2. Use of a commercial interferometer. FISBA also
makes available the on-axis and off-axis PSFs.
– Both methods give accurate results when compared
with the data from the lens manufacturer.

Slide 10
The Measurement of the Intensity Imaging, Robotics, and Intelligent Systems

PSF for Cell Phone Cameras IRIS

• High resolution PSF of the lens

1. The point source object is at a finite distance. A pinhole is

illuminated by a white light LED, and the created image by the lens
is magnified by the Olympus microscope and imaged onto a CCD
detector. The lens under test is fixed after alignment.

Slide 11
The Measurement of the Intensity Imaging, Robotics, and Intelligent Systems

PSF for Cell Phone Cameras IRIS

• High resolution PSF of the lens
2. By using FISBA(interferometer) with some minor
modifications, the authors were able to measure both
infinite conjugate distance and finite conjugate distance.
FISBA uses a helium neon laser illumination. The measured
high resolution PSF is shown.

Slide 12
The Measurement of the Intensity Imaging, Robotics, and Intelligent Systems

PSF for Cell Phone Cameras IRIS

• Averaged Down Sampled System PSF
– The sampled-scene phase parameter is the relative
location of the point source to the sampling grid.
In this case the sampling grid has size of one pixel
of CMOS sensor.
– The overall system PSF is given by
1 𝑥 𝑦
ℎ𝑠 𝑥, 𝑦; 𝑥0 , 𝑦0; 𝑥𝑟 , 𝑦𝑟 = ℎ 𝑥 − 𝑥𝑟 , 𝑦 − 𝑦𝑟 ; 𝑥0 , 𝑦0 ⊗ 𝐴(𝑥, 𝑦) 𝑐𝑜𝑚𝑏 ,
∆2 ∆ ∆
– Where ℎ 𝑥 − 𝑥𝑟 , 𝑦 − 𝑦𝑟 ; 𝑥0 , 𝑦0 is the digitally shifted
high resolution PSF of the lens.
– Assumption: the response over the active pixel
area is uniform and the fill factor is very close to
𝑥 𝑦
100%, then 𝐴 𝑥, 𝑦 = 𝑟𝑒𝑐𝑡 ,
△ △
Slide 13
The Measurement of the Intensity Imaging, Robotics, and Intelligent Systems

PSF for Cell Phone Cameras IRIS

• Averaged Down Sampled System PSF
– The overall system PSF on the Bayer plane

1 𝑥 𝑦
ℎ𝑠 𝑥, 𝑦; 𝑥0 , 𝑦0 ; 𝑥𝑟 , 𝑦𝑟 = ℎ 𝑥 − 𝑥𝑟 , 𝑦 − 𝑦𝑟 ; 𝑥0 , 𝑦0 ⊗ 𝐴(𝑥, 𝑦) 2 𝑐𝑜𝑚𝑏 ,
4∆ 2∆ 2∆
– 2 △ is the spacing between two adjacent pixels with
the same Bayer filter.

To illustrate how the system PSF is shift variant with

the point source location relative to the sampling
grid, two down-sampled system PSFs are shown.

Slide 14
The Measurement of the Intensity Imaging, Robotics, and Intelligent Systems

PSF for Cell Phone Cameras IRIS

• Averaged Down Sampled System PSF
– To the right, the down sampled system PSF, the
point source is located at the center of one pixel.
To the left, the point source was located at the
edge of one pixel.

Slide 15
The Measurement of the Intensity Imaging, Robotics, and Intelligent Systems

PSF for Cell Phone Cameras IRIS

• Averaged Down Sampled System PSF
– The figures show the cross section along the x-axis of the
PSFs shown in previous slide. To the right, the point source is
located at the center, to the left the point source is located at
the edge of two pixels.

Slide 16
The Measurement of the Intensity Imaging, Robotics, and Intelligent Systems

PSF for Cell Phone Cameras IRIS

• Averaged Down Sampled System PSF
– To obtain a robust PSF that does not depend on the relative
location of the point source to the pixel, the authors assumed
that the relative location of the point source is uniformly
distributed in the interval.
∆ ∆ ∆ ∆
− ≤ 𝑥𝑟 ≤ , − ≤ 𝑦𝑟 ≤
2 2 2 2
– The average of the sub-pixel shifted high resolution PSFs is
used to obtain the average high resolution PSF which is given
by
△ △
σ 2 △ σ 2 △ ℎ(𝑥 − 𝑥𝑟 , 𝑦 − 𝑦𝑟 ; 𝑥0 , 𝑦0 )
𝑥𝑟 =− 𝑦𝑟 =−
2 2
ℎ𝑎𝑣𝑒 𝑥, 𝑦; 𝑥0 , 𝑦0 =
100

Slide 17
The Measurement of the Intensity Imaging, Robotics, and Intelligent Systems

PSF for Cell Phone Cameras IRIS

• Averaged Down Sampled System PSF
– The averaged system PSF on the CMOS sensor
plane is shown.

Slide 18
The Measurement of the Intensity Imaging, Robotics, and Intelligent Systems

PSF for Cell Phone Cameras IRIS

• Averaged Down Sampled System PSF
– The figure summarizes all four PSFs obtained so
far in this paper.

– The measured overall PSF is broader than the high

resolution PSF lens only.
Slide 19
The Measurement of the Intensity Imaging, Robotics, and Intelligent Systems

PSF for Cell Phone Cameras IRIS

• Averaged Down Sampled System PSF
– The authors further calculate the down sampled system PSF
on the Bayer plane. The PSF is shown the right, and the
corresponding cross sectional area is shown to the left.

– The down sampled PSF on Bayer plane drops in intensity

much more than the down sampled PSF sensor plane.

Slide 20
 Image Restoration Results
Imaging, Robotics, and Intelligent Systems

IRIS

– In order to compare the performance of the

different PSFs, the Wiener filter Algorithm was
applied to the G1 channel image on the Bayer
plane without any demosaic, de-noise and color
processing.
– The image was captured using a 3M cell phone
camera with an EDoF lens.
– The illumination light level is 400 lux and camera
was set to audio gain control

Slide 21
 Image Restoration Results
Imaging, Robotics, and Intelligent Systems

IRIS
• The figure shows the G1 channel raw image in the Bayer plane,
the restored image with the non-averaged PSF on the Bayer
plane, and the restored G1 channel image with the averaged
down sampled system PSF on Bayer plane in that order.

Slide 22
 Image Restoration Results
Imaging, Robotics, and Intelligent Systems

IRIS
• These images are cropped and the results
compared.

Slide 23
 Image Restoration Results
Imaging, Robotics, and Intelligent Systems

IRIS
• The restored image with the averaged down
sampled system PSF is the sharpest .

• The last cropped image does not show any

over contrast artifacts such as ringing effects.

• From the images, the authors conclude that

the averaged PSF for image restoration
compared with the non-averaged system PSF
produces better results.

Slide 24
 References
Imaging, Robotics, and Intelligent Systems

IRIS
1. C. Liu and X. Chen, “Point Spread Function
(PSF) Measurement for Cell Phone Camera
with a High Resolution PSF of the Imaging
Lens and a Sub-pixel Digital Algorithm.”
[Online]. Available:
http://144.206.159.178/ft/CONF/16419973/
16419985.pdf. [Accessed: 01-Feb-2013].

Slide 25