2010 International Conference on Computer Application and System Modeling (ICCASM 2010)

Design and Implementation of Image Fusion System

Zhihui Wang, Yang Tie, Yueping Liu

Department of Electronic Engineering, College of Electronic Information and Engineering
Inner Mongolia University, Hohhot 010021, China
E-mail: [email protected]

Abstract-Image fusion is one of important image processing Section III explains the design rules and design steps of
technologies. Multi-wavelets, Pulse Couple Neural Network Graphical User Interface (GUI). In section IV, we implement
(PCNN), and fractal dimension are very useful theoretical tools an image fusion system based on GUI. Section V gives the
for image processing. We introduce the basic principles of experimental results.
image fusion at the level of pixel, feature, and decision. We
describe the design rules and steps of Graphical User Interface II. PRINCIPLES OF IMAGE FUSION
(GUI). Image fusion system based on GUI is then designed
Pixel-level image fusion is the information fusion that is
and implemented. We give the framework of the overall design
implemented directly using the basic data of the images
of the system and explain its usage method. The system has
many functions: image denoising, image enhancement, image
needed to be fused. It is image fusion at the lowest level that
registration, image segmentation, image fusion, and fusion has the highest degree of integration accuracy. Pixel-level
evaluation. Especially can the system implement image fusion fusion is able to provide the detail information of image that
based on wavelet, or multi-wavelets, or PCNN, or Match can not be brought by those at the other levels. Of course, it
Measure of Pulse Number (MMPN). In order to show that the should be noted that pixel-level fusion requires the largest
system is intuitive and easy to use, we apply it to visible image amount of information. Its structure is shown in Fig.l.
and IR image. It is worthy to be emphasized that pixel-level image
fusion integrates the information of multi source images on
Keywords-image fusion system; multiwavelets; peNN; GUf; the premise of strict registration.
fusion evaluation. Feature-level fusion belongs to the information
integration processed at middle level. It not only retains
I. INTRODUCTION sufficient important information, but also compresses the
information. So it is beneficial to real-time processing. We
Image fusion is one types of information fusion. It fuses
can find the difference between pixel-level processing and
several images from different image sensors in order to
feature-level one from Fig. I.
obtain a new image that contains more information and has
In fact, the features extracted from multi source images
more positive image description to the same scene. So far
are synthesized and processed in feature-level fusion.
image fusion has been widely used not only in some military
The structure of decision-level fusion is shown in Fig.2.
fields, such as object detecting and tracing, context
awareness and so on, but also in many civil fields which
include navigation of airport, security control, intelligent
traffic, geographical information system, medical imaging,
and human being visual aids and so forth [1].
Domestic fusion research started in the nineties. After the
Gulf War, information fusion technology caused great
attention in China. Since then, some universities and research
institutions have conducted research work in this area, but Other Follow-up Image Processing

most are the theoretical exploration. At present, image fusion

Figure I. Structure of image fusion at pixel level and feature level
study in China is still in a backward state, especially in the
practical aspects of engineering research [2].
We design and implement an image fusion system, in
which you can fuse multi source images by traditional
wavelet, multi-wavelets [3], and Pulse Couple Neural
Network (PCNN) [4] at image pixel level. It is well known
that image fusion can be run at three different levels: pixel
level, feature level and decision level. Of course, you can
also decide to process image fusion at feature level or
decision level. Up to now there is little image fusion system
that can fuse images basing on wavelet, multi-wavelets, and
PCNN. The paper is organized as following. Section II
reviews the principles of image fusion at different levels. Figure 2. Structure of Decision-level image fusion

978-1-4244-7237-6/10/$26.00 mOl 0 IEEE VIO-140

 2010 International Conference on Computer Application and System Modeling (ICCASM 2010)

Such integration is to make an optimal decision based on multi-scale and multi resolution. However, pyramid-based
some certain criteria and the reliability of each decision. It decomposition approaches have no direction, and wavelet
is the highest level fusion and the most complex one. There system can not simultaneously have orthogonality, symmetry,
is lower requirement of image registration in decision- level smoothness, compact support, vanishing moments that are
fusion, even without considering. very important in image processing.
Fusion technologies at different levels have their own Multi-wavelets is an expansion of traditional wavelet and
advantages and disadvantages. Among them, pixel-level has more advantages. It is most important that a multi­
fusion has been gotten widespread attention because of the wavelets system can simultaneously have these
most abundant information. characteristics that are preserving length (orthogonality),
good performance at the boundaries because of linear-phase
III. GRAPHICAL USER INTERF ACE symmetry, and a high order of approximation also named
Graphical User Interface (GUI) is a user interface vanishing moments. Thus, multi-wavelets transform offers
consisting of window, cursor, keys, and menu, etc. The user the possibility of superior performance for image fusion,
selects or activates these graphic objects through a certain compared with traditional scalar wavelets. There was
method so as to make the computer completes certain actions, someone began to use multi-wavelets for image fusion in
such as achieving a certain algorithm or drawing. 2001 [7].
On the other hand, the neural network algorithms have
A. Design principles ofGUl their significant advantages in image feature analysis. Pulse­
In short, a good user interface should follow three Coupled Neural Network (PCNN) is a simplified neural
principles that are simplicity, consistency, and familiarity network model that was constructed by Eckhom [6] who
[5]. studied the principle of the cat cortex in 1989. Broussard
Simplicity means that designer should reflect the applied PCNN to image fusion firstly in 1999. PCNN has its
functions and features of the user interface simply, directly inherent superiority to other traditional image processing
and clearly. algorithms due to its strong biological background.
As for consistency, it requires that the new interface We design an image fusion system which can implement
should be consistent with other existing interface, by which image fusion based on wavelet, or multi-wavelets, or PCNN,
the same thing is handled like the new one. or MMPN [8] through GUi. It can also realize feature-level
When designing a new interface, designer should make fusion or decision-level fusion.
full use of the familiar signs and symbols. This is the rule
A. Overall Design of Fusion System
advocated by familiarity.
From the functional point of view, the image fusion
B. Design Steps ofGUl system can perform image denoising, image enhancement,
In order to achieve a GUI with good performance, the image registration, pixel-level image fusion, feature-level
specific design steps are as follows: fusion, and decision-level fusion, etc. Moreover, it can also
• Analysis the main function that the interface requires, quantitatively evaluate the performance of fusion algorithm
determine the design task. by calculating image entropy, standard deviation, average
• Draw an outline sketch of the interface and examine gradient, spatial frequency, and quality measurement [9] and
it standing the poison of users. so on.
• According to the sketch, design static interface Fig.3 is the Start Interface of the image fusion system.
firstly and check it carefully. When pushing the Enter pushbutton, we can enter the main
• Write the dynamic program through implementing interface shown in FigA. There are 8 main menus: File,
the corresponding callback functions of the objects Image-Denoising, Image-Enhancement, Image-Registration,
on the interface. Image-Segmentation, Image-Fusion, Fusion-Evaluation, and
It is an unavoidable fact that the design steps maybe need Help. Four image showing regions are designed on the
to be modified several times so as to get a satisfY interface interface. Two left regions are used to show the two original
finally. images needed to be fused respectively. In order to compare
the fusion results and of different al thms,
IV. IMAGE FUSION SYSTEM

In recent years, we have been focusing on the pixel-level

image fusion. We have proposed some fusion algorithms.
The approach of image fusion at pixel level can be
divided into simple method, pyramid decomposition, wavelet
decomposition, etc. Simple method includes largest value
selection, smallest value choice, average value selection, and
entropy choice. Although these methods have lower degree
of complexity, they reduce the entropy value and the contrast
of the image [2]. The fusion method based on pyramid
decomposition or wavelet decomposition can stress the
important characteristics and detail information of image in Figure 3. Start Interface of image fusion system

VIO-141
 2010 International Conference on Computer Application and System Modeling (ICCASM 2010)

.....
1 "£",,,

� h... l., c...fflcl ... .-Ba.tod hJh'od- ....... ,. ,... 1... '... ur ... .... ...
I I' I""

..".1.' Conu ... .

-s. tod ,"',"" [1•..:101 __ 1_1 'Ull""

c::::::J
IS'.�d",,,,,·li lul.I .. ... .
l .. . Con.ra . -Ba• .d'''''lon
lul.ln ... ... ... .... . ...
l." ct. .ur a. .<l1 lon

c::::::J lul,I .. ... .

l .. ..
andI'CllI-B .d,... ion

I
lmagl'. A to be fused �
Fust'd Image 1 b L-____---'

Image B to be fused Fused Image 2 by I Image B to be fused Fused Image 2 by I

Figure 4. Main Interface of image fusion system Figure 7. Image-Fusion menu

we can observe the fused images obtained by two different To those images that are not registered, we should
algorithms in the right two areas on the interface. In addition, perform registration by clicking the Image-Registration
we put three most common evaluation indices that are menu before image fusion. Of course, two source images
entropy, standard deviation, and quality measurement on should be loaded firstly through File menu before all other
the interface directly as in Fig.4. operations. when selecting File/Load Image menu, the
appearances of the interface is shown in Fig.6.
B. Using the Fusion System
Because image fusion is the key operation of the system,
If the source image is corrupted one, we can implement we introduce the image fusion processing emphatically. As
denoising operation by some appropriate method listed in described previously, image fusion can be implemented at
Image-Denoising menu as Fig.5. three different levels: Pixel-level, Feature-level, and
Decision-level.
Now taking the case of pixels-level fusion, we introduce
image fusion operation. According to Fig.7, we can find
lo.ch 'Uteri", that there are four types of algorithms in pixel-level fusion
o I�
domain: Simple method (Pixel Values Weighted-average
Fusion), Wavelet-based methods, Multiwavelets-based
methods, PCNN algorithms, and Fractal dimension-based
Inulge A to be fused J1
I<'used Image I b L____---'
method [7] [8] [9] [10] [11]. When using multiwavelets­
based algorithm, we can choose what kind of multiwavelets,
such as OHM or CL.
In addition to entropy, standard deviation, and quality
measurement, there are the other evaluation indices that can
be adopted. It can be realized by the Fusion-Evaluation
Image B to be fust'd Fused Image 2 by I menu shown in Fig. 8.

Figure 5. Image-Denoising menu

Image A to be fused Fused Image 1 b '---____

-- ...J

htHlge A to be fused �
Fused Image 1 b L-____---'

Image B to be fused Fused Image 2 by I

Inmgc B to be fused Fused Image 2 by I Figure 8. Fusion-Evaluation menu

Figure 6. File->Load image menu

VIO-142
 2010 International Conference on Computer Application and System Modeling (ICCASM 2010)

V. EXPERIMENTAL RESULTS VI. CONCLUSIONS

We apply the system to several groups IRiVisible images. The image fusion system designed on QUI environment
Firstly, two source images are loaded by selecting can integrate images at pixel level, feature level, or decision
File/Load Image menu. The result is shown in Fig.9. level. It makes use of many advanced image processing tools:
Assuming that the original images have been registered, the multi-wavelets, PCNN, and fractal dimension. The system,
fusion can be operated subsequently. which has image denoising, image enhancement, image
We can easily evaluate the fusion result from the angle registration, image fusion and fusion performance evaluation
of sUbjectivity and objectivity at the same time. Moreover, function, is intuitive and easy to use.
the data of three common indices are directly shown on the
interface just like Fig.10. Although the same fusion rule ACKNOWLEDGMENT

constructed with Match Measure is used, multiwavelets­ This work has been partially supported by Inner
based algorithm really offers the superior performance for Mongolia Natural Science Foundation Subject (No.
image fusion, compared with traditional scalar wavelets. 2009BS0903), Higher Education Scientific Research
Due to limited space of the paper, we did not give the Projects in Autonomous Region (No.NJ09005), and the
feature-level fusion result and decision-level one. Scientific Research Startup Foundation Subject of
Introduction of High-level Talent of Inner Mongolia
University (No. 208045).

REFERENCES

[I] Z.L. Jing, GX, and Z.H. Li, Image Fusion: Theory and Applications.
Bei Jing, China: Higher Eduction Press, 2007.
[2] W.G. Liu, Image Fusion and Recognition. Bei Jing, China: Publishing
Image A to be fused House of Electronics Industry, 2008.
[3] V. Strela, Multiwavelets: Theory and Applications. Massachusetts,
USA: Massachusetts Institute of TechnoJogy, 1996.
[4] R.P. Broussard, S.K. Rogers, and E. Mark, "Physiologically
Motivated Image Fusion for Object Detection using a Pulse Coupled
Neural Network," IEEE Trans on Neural Networks, vol. 10, pp.554-
563,1999.
lnuige B to be fused lnHtge U to b(' fused [5] Z.Y. Zhang, Proficient MATLAB6.5 version. Bei Jing, China:
Beijing University of Aeronautics and Astronautics Publishing House,
(a) (b) 2003.
[6] R. Eckhorn, H. J Reitboeck, M. Arndt, etc. A neural network for
Figure 9. Loading source images: (a)VisibIe image. (b)IR image
feature linking via synchronous activity: Results from cat visual
cortex and from simulations. Models of Brain Function. R.M. J
Cotterill, Ed. Cambridge, U. K. : Cambridge Univ. Press, 1989,
t .....
.. A Eno: pp.255-272.
[Elllro".1 I EIltfOIlYI
[7] Z.H. Wang, B.J. Zhao, and T.Z. Shen, "Novel Image Fusion Method
I 9A622 I � using Multiwavelet Transform," Proc of SPIE. Multispectral Image
Isu,,,.. ..111...-I·1
Processing and Pattern Recognition 2007(MIPPR 2007), OF-I- OF-9.
I 77·U2 I [8] Z.H. Wang, B.J. Zhao, and T.Z. Shen, "Image Fusion Based on
..
I 0 ,,,, I
MMPN and Adjustable Link Strength,"ACTA Electronica Sinica, vol.
� 38, pp. 1162-1166,2010.
[9] Z.H. Wang, Study of Image Fusion Algorithms Based on
lin... a En, Multiwavelet Transform. Bei Jing, China BEIJING INSTITUTE OF
[Elllro", I
TECHNOLOGY,2008.
� [10] Z.H. Wang, Y. P Liu, and Y. Tie, "Image fusion based on fractal
theory in multi-wavelet transform domain," unpublished.

.. [II] Z. H. Wang, B.J. Zhao, and T.Z. Shen, "Image Fusion Based on
I 0 ,,,, I
I O.80�61 Match Measure and Multiwavelet Transform," Optical Technique,
vol. 34, pp.146-148, 2008.
Image B to be fused

Figure 10. After image fusion

VIO-143