World Academy of Science, Engineering and Technology 32 2009

Blind Source Separation Using Modified Gaussian FastICA

V. K. Ananthashayana, and Jyothirmayi M.

AbstractThis paper addresses the problem of source separation

in images. We propose a FastICA algorithm employing a modified Gaussian contrast function for the Blind Source Separation. Experimental result shows that the proposed Modified Gaussian FastICA is effectively used for Blind Source Separation to obtain better quality images. In this paper, a comparative study has been made with other popular existing algorithms. The peak signal to noise ratio (PSNR) and improved signal to noise ratio (ISNR) are used as metrics for evaluating the quality of images. The ICA metric Amari error is also used to measure the quality of separation.

KeywordsAmari error, Blind Source Separation, Contrast function, Gaussian function, Independent Component Analysis. I. INTRODUCTION

LIND source separation (BSS) problem has become an important application in the research domain of statistical signal processing. In a basic blind source separation problem, the goal is to separate mutually statistically independent sources, without knowing the mixing coefficients from their instantaneous linear mixtures. But verification of independent condition of these sources is very difficult. Hence some higher order statistics are used in practice for achieving separation. Independent Component Analysis (ICA) is one of the statistical tools that use higher order statistics to discover the hidden factors from the set of observed data. BSS technique[5,6,8] has a variety of applications in communication, speech processing, multimedia and medical image processing [1, 2]. Mathematically, the BSS problem can be formulated as

separability of the mixed sources depends on the principal of redundancy reduction and the sources must be as independent as possible. In this paper, the popular fixed point algorithm with a new objective function called Modified Gaussian function to extract the independent components is used. The objective function is based on the approximation of negentropy [4] that needs to be maximized. The statistical property of this function is demonstrated for its robustness in case of image mixtures. The images considered are here can be compared with the work in [9]. Further, we measure the quality of the separated images with Amari metric and assess the performance of separated images using ISNR and PSNR. The remainder of this paper is organized as follows. In section II, the methodology for source separation using ICA is discussed; section III gives results and conclusion in section IV. II. METHODOLOGY A. ICA Model ICA is formulated as a problem by which independent original sources are extracted based on the observations of mixture sources. The model can be expressed as a linear combination (Fig. 1.)
s 1 (t ) M a 11 (l ) L M M O a M 1 (l ) L M s N (t ) x1 (t ) M a 1 N (l ) w11 (l ) L M M M O w N 1 (l ) L a MN (l ) M x M (t ) y 1 (t ) M w1 M (l ) M M w NM (l ) M y N (t )

Fig. 1 The schematic diagram of BSS

X = WS + n

(1) The linear equation for two source images can be expressed as:
X
1

where x is the observed image (mixed), S is the original source images obtained which is transformed by the mixing matrix W whose elements are unknown and n is the additive noise often omitted. Therefore, a multivariate data analysis, such as ICA [3, 4, 11,12,13], might be used to solve the source separation and restoration problem. In ICA, the

(t ) = (t ) =

a 11 S 1 + a 12 S 2

a 21 S 1 + a 22 S 2

(2)

V. K. Anathashayana is presently working as Professor and Head, with the Department of Computer Science and Engg, M.S.Ramaiah Institute of Technology, Bangalore, Karnataka, India. Jyothirmayi.M, Assistant professor, is with Department of Instrumentation Technology, M.S.Ramaiah Institute of Technology, Bangalore, Karnataka, India (e-mail: [email protected]).

where S1 and S2 are original sources, X1 and X2 are observed (mixture) sources and a is the unknown matrix that needs to be estimated. By determining a, we can separate the two original signals S1 and S2 from their mixtures X1(t) and X2(t). The number of sources N (known) can be estimated, when the number of sensors M is equal to or greater than N, M N . The mixed signal X is represented as in (3). This ICA model is a generative model that describes the computation of S by

394

World Academy of Science, Engineering and Technology 32 2009

estimating the mixing matrix a from the known random vector denoted by S = A 1 X and W = A 1
x1 M M M xM a 11 = M a M 1 L O L s1 a1N M M M a MN M sN

(3)
Fig. 3 Probability distribution functions showing the variation of K value (K=0.3, k=0.01, K=0.9)

To estimate W some general and fundamental assumptions are made. a) The components sN (i.e. the sources) are statistically independent. b) The independent components must have nongaussian distribution or utmost one source may have gaussian distribution. (Fig. 8) Depending on the application, the sources and the mixture may be real or complex. Source separation is carried out by estimating a separating matrix W such that the separated output Y=WX contains an estimate of the sources. To satisfy the assumption (a), the mixed signal is first whitened [10] thereby restricting the search space for the mixing matrix to the space of orthogonal matrices. To measure the non gaussianity, kurtosis or differential entropy called negentropy can be used. Since kurtosis is not a robust measure of nongaussianity, we consider negentropy. The function [4] JG , a maximizing entropy equation can be used as a gradient approximation of negentropy
J G (w ) = E G ( w

Estimation of W is a measure of minimization of mutual information and maximization of negentropy which can be interpreted as estimation of single Independent component. In this section, we describe our approach based on the objective function The Modified Gaussian function, abbreviated as MGFICA. The unmixing matrix W is estimated by maximizing the contrast function with respect to W given by:
W = g ( w j xi )
T i =1 j =1 n p

(6)

where g ( y ) is the modified Gaussian function, defined by

g ( y ) = exp( ( y )
2

( 2 K ))

(7)

[{

x E {G (

)}

)}

(4)

where G is a non quadratic function, v is a Gaussian variable of zero mean and unit variance and w can be found under the constraint E {(wTx)2}=1 B. Contrast Functions The First thorough mathematical framework for BSS and ICA was established by Comon in [6]. In order to achieve separation, it is important to choose a non-linearity function g, a derivative of G [4, 7, 8, 14]. The objective function used in FastICA is

g ( y ) = y exp y

where, K is a parameter that gives a measure of distribution, (Fig. 3). By varying the parameter K, it is possible to describe the function as Gaussian, platykurtic, and lepokurtic distributions that control the spread of the distribution. The plot of a contrast function defined by A.Hyvarinen [4] is shown as a comparison with the modified Gaussian plot Fig.2 for K=0.1. Optimal value of K can be found from experimentation. The constant parameter gives the algorithm a kind of Self adaptation quality. From the distributions it is clear that, for certain values, the probability distribution function (pdf) is spiky and hence the entropy will be small. Hence it is possible to choose an efficient contrast function based on the probability spread function that has maximum entropy, converges faster and the sources are separated with less distortion. C. Modified Gaussian FastICA Algorithm (MGFICA) To simplify the algorithm, the data X is centered and whitened by computing covariance E{xxT}. FastICA[10] uses convergence criteria in which the old and the new values of W estimated must point in the same direction. To extract all the independent components one by one, we need to run the algorithm several times. To prevent the vectors w1wN converge to same maxima we orthogonalize the vectors after every iteration. For the function g, we use a robust approximation of negentropy as in (7) and for the problem of BSS, we demonstrate that quality of separation is improved with our modified Gaussian function. Hence choice of the right function is also very important as a robust estimator. Table I provides the algorithmic description for the MGFICA.

2)

(5)

Fig. 2a gives the distribution spread for this equation. As g(.) is dependent on the estimated sources y the distribution obtained is sigmoid.

(a) (b) Fig. 2 Plot showing a) the Gaussian function for the FastICA b) The Gaussian function for MGFICA

395

World Academy of Science, Engineering and Technology 32 2009

TABLE I ALGORITHM FOR MGFICA

1. 2. 3. 4. 5. 6.

Center the data to make its mean zero. Whiten the data to give Z. Choose an initial random vector W of unit norm.

W E Zg (W Z )} E g (W Z )}W
T T

Normalize W W W

If not converged, go to step4.

Fig. 4 Original Images

III. EXPERIMENTAL RESULTS We demonstrate the capability of the proposed algorithm to successfully separate four images and compare its performance to competing algorithms. Four source images1 of size 256x512 are shown in Fig. 4. The pixel intensities are normalized in the [0,255] interval. The observed image obtained by mixing original images with a 4x4 random matrix resulting in a mixture of size 4x131072 Fig. 5. The separated image obtained after applying MGFICA algorithm is shown in the Fig. 6 and with FastICA is shown in Fig. 7. A comparison of Fig. 6 and Fig. 7 for source separation can be visualized. The variation of K for different values is also experimented and optimal value is found to be 0.3 for quality separation. For the evaluation of the quality of demixing matrix W, the Amaris error d(W,A), an Amari metric (Amari et al.(1996)) is used. The Amari error (8) is used as a measure to compare different ICA algorithms and the matrices should be scaled in the same way. The value so obtained is such that it is scaled to a value between 0 and 1 and 0 corresponds to optimal separation. Table II shows Amari error for comparison with other algorithms.
N 1 d (W , A ) = 2 N ( N 1) i =1 N N bij b N ij j =1 i =1 1 + 1 max j bij j =1 max i bij

Fig. 5 Mixed images

Fig. 6 Images Separated from MGFICA

(8)

where bij = (W-1A)ij, W and A are NxN matrices. For the purpose of objectively testing the performance of separation problem, the Peak Signal to Noise Ratio (PSNR) is often used (9). We also evaluate the performance based on Improved Signal to Noise Ratio (ISNR) as in (10). Results are tabulated in Table III and Table IV.
psnr = 20 log 10 s(m , n) y (m , n) 255

Fig. 7 Images separated from FastICA

dB

(9)

M N m =1 n =1

s (m, n) x(m, n) Isnr = 10 log10 s ( m, n) y (m, n) dB

(10)

http://www.cis.hut.fi/projects/ica/data/images/

Fig. 8 Histogram representation of mixed images

396

World Academy of Science, Engineering and Technology 32 2009

TABLE II AMARI ERRORS Algorithms MGFICA FASTICA with Gauss Function FASTICA with Tanh Function Radical ICA Amari Error between mixing and demixing matrix 0.3877985 0.4530318 0.4689753 0.5191616

[5]

[6] [7]

[8]

TABLE III COMPARISON WITH DIFFERENT METRICS PSNR in dB Fig No a b c d Fast ICA 12.13 11.06 16.33 10.61 Radical ICA 16.4401 17.7517 17.8795 17.3477 Modified ICA 26.33 30.43 10.492 23.83

[9]

[10] [11] [12] [13] [14]

TABLE IV COMPARISON WITH DIFFERENT METRICS Fig No a b c d Fast ICA -0.1 -3.0 3.6 1.5 ISNR in dB Radical ICA -1.0 1.09e03 1.05e03 1.1976e03 MGFICA -8.14 13.14 12.7 11.72

A.J.Bell and T.J.Sejnowski, An information-maximization approach to blind sep-aration and blind deconvolution, Neural computation,, 1995,7:1129-1159. P.comon, Independent component analysis, a new concept? Signal Processing, vol.36, no3, pp.287-314, Apr.1994. J.F.Cardoso, Infomax and maximum likelihood for blind source separation, IEEE signal processing letters, vol.4, April1997,pp. 112114. J.Karhunen, E.oja,L.Wang, R.Vigario, and J.Joutsensalo, A class of neural networks for independent component analysis, IEEE trans. On Neural Networks, Mat1997, vol.8,pp.486-504. Sergey Kirshner and Barnabs Pczos, ICA and ISA Using SchweizerWolff Measure of Dependence, Proceedings of the 25th International Conference on machine Learning, Helsinki, Finland,2008. Aapo Hyvrinen,Juha Karhunen and Erkki Oja, Independent Component Analysis, John wiley & sons, Inc 2001 Hyvrinen,A., Fast and robust fixed-point algorithms for independent component analysis, IEEE Trans. On Neural Networks, 626-634,1999. A.Cichocki and A. Amari, Adaptive Blind signal and Image Processing, John-wiley and sons,2002. Junhua Wang, An Image BSS Algorithm Based on Curvelet Transform, IEEE Transactrions, ICALIP2008, 2008. Te-Won Lee, Michael S.Lewicki, The Generalized Gaussian Mixture Model Using ICA, IEEE Acoustics,speech and signal processing, vol2,pp1161-1164,1998.

We estimate the demixing matrix W from which the sources are separated. This result is compared with the different algorithms like FastICA and Radical [9]. Table III, IV shows the quality assessment for all the separated Images which gives an improvement in PSNR and ISNR. IV. CONCLUSION The proposed Modified Gaussian ICA is used to solve the problem of Source Separation. The effectiveness of the algorithm is the selection of the negentropy based function that can give optimal solution. The simplicity of using a modified gaussian function is the main criteria in this algorithm. Experimental result shows that there is a better quality of separation of the mixed images (Amari error) and the PSNR has improved moderately than in the existing ICA algorithms. The visual quality as well as the objective quality in terms of ISNR of the reconstructed images can also be compared. REFERENCES
[1] J.Karhunen, A.Hyvarinen, R. Vigario, J.Hurri, and E.Oja, Applications of neural blind separation to signal and image processing, in Proc. 1997 IEEE Int. conf. on Acoustic, speech, and Signal Processing (ICASSP97), Munich, Germany, April 1997 pp-131-134. , E.Oja, J.Karhunen, A.Hyvarinen, R. Vigario and J.Hurri, Neural Independent Component Analysis Approaches and Applications, in Brain-like Computing and intelligent Information Systems, S.I. amari and N.Kasabov (Eds.), Springer-Verlag, Singapore,pp.167-188., 1998 S.I.Amari, A. Cichocki, and H.H.Yang, A new learning algorithm for blind signal separation, 1996, 8:757-763. A.Hyvarinen, Fast and Robust Fixed-point Algorithms for Independent Component Analysis, IEEE Trans. On Neural networks,vol.10(3),p.p 626-634, 1999

[2]

[3] [4]

397