Proceedings of the 2006 IEEE

International Conference on Robotics and Biomimetics

December 17 - 20, 2006, Kunming, China

A New Wavelet Based Multi-focus Image Fusion Scheme

and Its Application on Optical Microscopy
Yu Song, Mantian Li, Qingling Li and Lining Sun
Robotics Institute
Harbin Institute of Technology
Harbin, HeiLongJiang Province, FR. China
[email protected]
Abstract - Multi-focus image fusion is a process of domain [7-12] due to its multi-resolution analysis (MRA)
combining two or more partially defocused images into a new characteristic [2-3]. It processes the images with different
image with all interested objects sharply imaged. In this paper, image resolutions and the decomposed different frequency
after reviewing the multi-focus image fusion techniques, a bands indicate different level image information. The
wavelet based fusion scheme with new image activity level
measurement is presented. The proposed multi-resolution image
information of detail, texture and edge of an image presents
fusion technique includes three steps: First, multi-resolution as the high-frequency sub-bands of its wavelet decomposition.
discrete wavelet transform (DWT) is applied to obtain the Some parameters are involved in wavelet based image
wavelet coefficients of the source images. Then, applying fusion scheme, such as selection of wavelet basis, wavelet
proposed coefficients fusion scheme to the obtained coefficients, decomposition scale and wavelet coefficients fusion scheme.
the wavelet coefficients of the fusion image are reconstructed. Hereinto, wavelet coefficients fusion scheme is the key issue
Finally, the final fusion image is generated by applying inversed in wavelet-based fusion. There are two main wavelet
wavelet transform. In experiments, artificial defocused images coefficients fusion schemes: coefficients selection based
(by applying low-pass filter to the specified regions) are utilized wavelet coefficients fusion scheme and coefficients weighted-
to investigate the performance of proposed scheme and to select
suitable wavelet family and wavelet decomposition scales. Then average based wavelet coefficients fusion scheme. The
we apply the proposed image fusion scheme to optical weighted-average based method, which will blur the image
microscopy domain to solve the short depth of focus details, is not suitable for multi-focus image fusion due to its
characteristic of optical microscope. The experiments results average operation characteristic. The selection based method,
verify the validity of the proposed multi-focus image fusion which selects the 'best' wavelet coefficients, is a prefer
scheme. method. Choose maximum coefficients scheme, choose
maximum coefficients variance scheme and choose maximum
Index Terms - Image fusion, Discrete wavelet transform, coefficients gradient scheme are typical selection-based
Image activity level, Optical microscopy methods.
In this paper, a new selection based wavelet coefficients
I. INTRODUCTION fusion scheme for multi-focus microscopic image fusion
The inherent characteristic of optical microscope is purpose is proposed. The proposed fusion scheme combines
limited depth of focus (from sub-micron grade to sub- the sampling wavelet coefficient and its local gradient
millimetre grade), only when the observed object is located at features together. The experimental results verify that this
the focal plane of the optical microscope, it will be imaged new method is useful for multi-focus image fusion.
sharply. So it is impossible to get an image with all relevant The remainder of this paper is organized as follows: In
observed objects inside view-field of microscope are "in Section IL, the proposed multi-focus images fusion scheme is
focus" . But we can get some images, in which the observed introduced. Section III is the experimental research and
objects are partially focused by using auto-focusing technique application research. Finally, Section IV is our conclusions.
[ 1 ]. So to get a new image, which contains the best II. WAVELET BASED MULTI-FOCUS IMAGE FUSION SCHEME
information coming from the obtained partially focused
images and achieves all objects "in focus" (namely, all A. Reviewing ofDiscrete Wavelet Transform (DWT)[2-6]
observed objects sharply imaged), a multi-focus image fusion Wavelets are functions in a space L2(R) of a basic
scheme is required. wavelet function using dilations and translations. They are
Wavelet theory is related to multi-resolution analysis used to represent the local frequency content of the functions.
theory. Since it was invented in 1980, many theoretical and The basic wavelet should be well localized, and a wavelet
application researches have been conducted. In recent years, should have a mean equal to zero.
wavelet transform has been introduced in image fusion

This work is partially supported by China National Nature Science Foundation (Grant # 505C5056).

1-4244-0571-8/06/$20.00 C)2006 IEEE 401

 A multi-scale analysis is an increasing spaces sequence By using the Mallat rapid algorithm [5,6], the 2-D
{VjJJ that every vector in Vj is also contained in Vj+1.That is: wavelet analysis operation consists in filtering and down-
Vo0 VI C V2 C V3 ... (1) sampling horizontally using the 1-D low-pass filter L and
A simple basis for space Vj is given by the set of scaled high-pass filter H to each row in the image f(x, y), producing
and translated of the scaling functions y (x): the coefficient matrices JL and JH. Vertically filtering and
down-sampling follows, using the low-pass filter L and high-
(j.k(X) = 2 jl2 .y(2' .x-k) k e Z (2) pass filter H to each column in JL and Jf and produces four
The orthogonal complement of Vj in V1+l is defined as sub-images LL, LH, HL and HH for one level of
wavelet space Wj. That is, Wj is the space of all functions in decomposition. LL is a smooth sub-image corresponding to
scaling space VT+l that are orthogonal to all functions in Vj. Wj the low-frequency band of the MRA and can be considered as
contains the details in VTjl that cannot be represented in Vj . a smoothed and sub-sampled version of the original image f
The basis functions V j,k (X) of wavelet space Wj have two (x, y), it represents the coarse approximation of original
important properties: image. LH, HL and HH are directional detail sub-images,
4 Every basis function jk (X) of W) is orthogonal to which represent the horizontal, vertical and diagonal
directions of the original imagef(x, y). Fig. 1 explains the one
every basis function 9 j I (x) of Vj. level rapid 2D wavelet transform, and Fig.2 is the example of
4-K The basis functions . j,k (X) of W) , together with the one level rapid 2D wavelet transform.
basis functionspjq, (x) of VjI, form a basis for V1/j,
In forward wavelet analysis, a function ftx) with J-level
wavelet decomposition is written as: Filtering and Filtering and
J down sampling down sampling
along the row [long the column
fl(X) = Wf (jo k) *f
, (X) + ,E W, (jo,^k) * fj k (X) (3)
k j=jo k
Fig. 1 Image wavelet transform with Mallat algorithm
The decomposition coefficients can be obtained by using
Mallat rapid algorithm [5,6] with filtering and down
sampling. The coefficients with resolution j are related to the
coefficients with resolution j+1 by the following recursive
equations:
W9 (j,k) L(- n) * W9 (j + 1, n)|n=2k k 0 (4)
WyX ( j, k) =H (-n) * W,, ( j + 1, n)| n=2k ,k20 (5)
Where, L is low-pass filter and H is high-pass filter, they satisfy
the relationship as:
H(n) (( -1) L(1- n) (6)
With the one-dimensional scaling function , (x) and Lena image Wavelet sub-bands
corresponding wavelet yr (x) , we can give separable two- Fig.2 Lena image and its one level wavelet transform
dimensional scaling and wavelet functions:
y (x, y) = y (x)y (y) B. Proposed method
After using wavelet transform technique decomposes the
Vf H (X, Y) =fl(x)(p(y)
source images, a wavelet coefficients fusion scheme is
,f (x, y) = f (x) f (y)
v (7) required to reconstruct new output image's wavelet
Vf D (X, Y) = (X*l (y) coefficients. To select the 'best' wavelet coefficients from the
Where, the symbols H, V and D stand for the directional wavelet. source images, an image activity level measurement is
The transformation basis finctions are defined as: designed. The proposed coefficients selection scheme
,jm.n (XIY) 212 y (2 jx - m2 J'y n) (8) considers the sampling wavelet coefficient and its local
j,mn(X,y) 2'2~'j12(2jx -m,2jy- n) i {H,V,D} gradient features together.
Thus, the resulting two-dimensional wavelet transform can be
used in an imagef(x ,y) of size M x N:
M*)N
W (9-1
-1
2=1 2=2
--12-1 P
W, ( jo, m, n) E f (xI y) V j,,m (xI y) (9) -1 -1
E

=
n

I M -1N-1
(jo, m, n)
WH fE f(X, Y)/f P-I ~ P-2 P-3 P-4
Ad[1t
oT0=
*N to,m,n (X, y)

i= IH,V,DI Fig.3 Four directional convolution kernels

402
 Let W A jk (m, n) and W B j,k (m, n ) stand for the III. EXPERIMENTS AND APPLICATIONS
wavelet coefficients of source image A and source image B A. Simulation Experiments
respectively (j is the decomposed resolution level and k= To investigate the influence of the wavelet basis function
0,1,2,3 is the four frequency bands). and the wavelet decomposition scale on the performance of
The local gradient of wavelet coefficient Wjjk(m n) is the proposed multi-focus image fusion scheme, the Lena
defined as below: image is utilized. Fig.5 (A) is the Lena image with left part
Gradient(Wjk (m, n)) = max Kp * Wjk (m, n),p =1 -4 (10) self-generated defocusing using mean filter technique and
Where, Kp are the four directional gradient operators Fig.5 (B) is the Lena image with right part self-generated
(p=l is the convolution kernel for 135 degree direction, p=2 defocusing using mean filter technique. And Fig.5 (C) is
is the convolution kernel for 90 degree direction, p=3 is the fusion result.
convolution kernel for 0 degree direction, p=4 is the
convolution kernel for 45 degree direction). The four
convolution kernels are list in Fig.3.
The proposed image activity level measurement
combines the wavelet coefficient at sampling point (m.n) and
its local wavelet coefficient gradient feature together, it is
defined as:
A(WJk(m,n)) =Gradient(WJk(m,n)) *WJk(m,n)| (11)
Where, A(Wjk(m,n)) reflects the activity level
information of the wavelet coefficient Wj,k(m,n).
The image fusion scheme based on proposed image (A) Lena image with left (A) Lena image with right
activity level measurement is expressed by (with the case of part is defocused part is defocused
two images fusion):
WWAj,k (i, n) A(WAj,k (m, n)) > A(WB j,k (m, n))
W°ut ;k (M,n) (12)
!WBj,k (m, n) A(WAj,k (m, n)) < A(WB j,k (m, n))
The multi-focus images fusion scheme based on
proposed method is illustrated in Fig. 4.
4,- Step 1: Discrete wavelet transform (DWT) with the
same decomposition scale is applied to obtain the
wavelet coefficients of the source images A and B
respectively.
4-K Step 2: Proposed wavelet coefficients fusion scheme is
carried to reconstruct new image's wavelet coefficients, (C) Fusion image
which contains the best information coming from the Fig.5 Self-generating partly defocused image and fusion result
source coefficients.
4-K Step 3: The reconstruct image wavelet coefficients are The root-mean-square (RMS) error criterion [13] is
adjusted using window-based consistency verification adopted for the objective evaluation of the fusion result. With
[1 1]. smaller RMS indicates better fusion result.
4 Step 4: The last output image is generated by applying I M-1IN-1
inversed wavelet transform (IDWT) with reconstructed
wavelet coefficients.
RMS=[ 1 M iA(i,j)l
ZZ[Ii=(ij=) if (il Al2 1Y'2 (13)
Fused wavelet Where, Ip is the ideal image and If is the fused image.
Proposed coefficients
Image A 1
coefficients TABLE I
fusion IMAGE FUSION WITH BIORTHOGONAL WAVELET FAMILY
method with Fusion RMS
window- 1EV- image (1,3) (2,2) (3,3) (4,4) (5,5)
based I n I rw let
consistency 1 4.7755 4.1906 2.7804 3.3370 3.6165
verification Inversed wavelet transform 2 3.7852 2.4350 3.0739 1.6271 1.7611
3 3.8168 2.4751 2.1027 2.5298 1.8709
Scale 4 3.3826 1.8008 3.2038 2.7284 1.9622
Wavelet transform
5 3.5607 1.8057 3.9617 2.9756 3.3415
6 3.6040 1.9839 4.1293 3.1352 2.6785
Fig.4 Wavelet based multi-focus image fusion scheme

403
 The proposed method with the Biorthogonal wavelet stage is focused. Combining the two images using proposed
family and Daubechies wavelet family are evaluated in method, a new image (Fig.7 (c)), which displays all
experiments. The evaluation results list in table I and table II. information under microscope, is obtained.
From the results, we can find that the proposed multi-
focus image fusion method with Db4 wavelet basis and 5
levels decomposition has the smallest RMS, namely, it is the
best one.
TABLE II
IMAGE FUSION WITH DAUBECHIES WAVELET FAMILY
RMS
Dbl Db2 Db3 Db4 Db5
1 4.3623 3.6217 3.4052 3.4130 3.5025
2 3.2946 2.7286 3.4390 2.4329 2.0208
3 3.6769 2.2840 3.3377 2.0639 2.9991
Scale 4 3.8810 2.2298 2.0868 2.1437 1.8637
5 4.0480 2.3152 2.0846 1.6134 1.9485 (a) Sharply micro gear image (b) Sharply background image
6 4.0930 2.4359 2.1682 1.6720 1.9041

Table III is the comparing results between the proposed

method and some existing method, such as choose maximum
wavelet coefficients method (CCM), choose maximum
wavelet coefficients variance scheme (CMV) and choose
maximum wavelet coefficients gradient scheme (CMG).
Besides the RMS criterion, PSNR (peak-signal-noise-ratio) is
also adopted as an evaluation criterion.
The PSNR is defined as below:
M N
ZZ2552
PSNR 1O lg( 101 ) (14) (c) Fusion result

OE (I(i j) If (i j))2 Fig.6 Micro gear images fusion

i=o j=o

TABLE III
IMAGE FUSION METHODS CONTRAST
CCM CMV CMG Proposed
RMSI4 2.3453 1.8725 1.9669 1.6134
PSNR 40.7610 T 42.7165 42.2892 44.0101

The comparing results show that the proposed fusion

scheme has the maximum PSNR value and the smallest RMS
value. It means that the proposed method is the best one
among those schemes.
(a) Sharply measuring block image (b) Sharply background image
B. Application on Optical Microscope Image Fusion
The motive of this research is solving the problem of
short depth of focus of the optical microscope. So two simple
applications are done to verify the validity of the proposed
method.
Fig.6 (a) is a sharply micro gear microscopic image, but
the background is defocused. Fig.6 (b) is a defocused micro
gear image with focused background. By carrying out the
proposed image fusion scheme, we get a new image (shown
in Fig.6(C)) with the micro gear and background both sharply
imaged.
Fig.7 is a measuring block with 0.5 mm thickness placed (c) Fusion result
on the microscopic object stage. In Fig.7 (a) the measuring
block is sharply imaged. In Fig.7 (b) the microscopic object
Fig.7 Measuring block images fusion

404
 IV. CONCLUSIONS
Main contributions of this paper are:
* After reviewing the wavelet transform and wavelet based
multi-focus image fusion methods, a new wavelet
coefficients selection based scheme is proposed.
* In experiments, the wavelet basis and decomposition
scale for proposed image fusion scheme are selected.
* We utilize the proposed image scheme to optical
microscopy domain. The results show that the objects
under microscope can be displayed to observer with one
fusion sharply microscopic image. It solves the inherent
problem of short depth of focus of optical microscope.
ACKNOWLEDGMENT
The authors gratefully acknowledge the valuable
contributions of anonymous reviewers of IEEE ROBIO 2006
conference for their helpful comments and suggestions
towards improving the manuscript.
REFERENCES
[1] Chern N. K., Nathaniel P. A. N., Marcelo H. A. J. "Practical issues in pixel-
based autofocusing for machine vision," IEEE International Conference on
Robotics and Automation, pp.2791-2796, 2001..
[2] I. Daubechies, "Ten Lectures on Wavelets", SIAM, Philadelphia, 1992.
[3] I. Daubechies, "Orthonormal bases of compactly supported wavelets,
Commun. Pure Appl. Math. 41 (1988) 909 - 996.
[4] M. Misiti, Y. Misiti, G. Oppenheim, J. Poggi, "Wavelet Toolbox for Use
with MATLAB: User' s Guide," The MathWorks, Natick, NA, 2000.
[5] Mallat S., "Multiresolurion Representation and Wavelets ". PhD Thesis,
University of Pennsylvania, Philadelphia, PA, 1988.
[6] S. Mallat, "A theory for multiresolution signal decomposition: the wavelet
representation," IEEE Trans. Pattern Anal. Machine. Intell, vol. 11, pp. 664-
693, Jul. 1989.
[7] V. Petrovic, C. Xydeas, " Multiresolution image fusion using cross band
selection, " Proc. SPIE 39 (1999) 319 - 326.
[8] H. Li, B.S. Manjunath, S.K. Mitra, "Multisensor image fusion using the
wavelet transform", Graphical Models Image Process. 57 (3) (1995) 235 -
245.
[9] Chibani, Y., Houacine, A. "Redundant versus orthogonal wavelet
decomposition for multisensor image fusion". Pattern Recognition. 2003
36, pp: 879-887.
[10]Gonzalo Pajares, "A wavelet-based image fusion tutorial", Pattern
Recognition. 2004 37, pp: 1855-1872.
[1 I]Simone G, Farina A, Morabito F C, Serpico S B, Bruzzone L "Image Fusion
Techniques for Remote Sensing Applications", Information. Fusion, 2002,
3(1), pp: 3-15.
[12]Wenzhong Shi ChangQing Zhu, Yan Tian, Janet Nichol "Wavelet-based
image fusion and quality assessment", International Journal of Applied
Earth Observation and Geoinformation, 2005 ( 6),pp: 241-251.
[13]R.C. Gonzalez, R.E. Woods, "Digital Image Processing," Addison-Wesley,
Reading, MA, 1993.

405