Scribd
Upload
70 views28 pages

Independent Component Analysis: Derek Beaton

Independent Component Analysis (ICA) is a statistical technique for separating mixed signals into independent non-Gaussian signals. ICA finds statistically independent components in mixed signals by reducing mutual information between outputs. It can be used for signal processing applications like the "cocktail party effect" as well as image processing tasks like separating overlapped or reflected images into their independent sources. ICA uses an unmixing matrix to perform a linear transformation on mixed data to maximize statistical independence between outputs.

Uploaded by

Gabriel Humpire
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
70 views28 pages

Independent Component Analysis: Derek Beaton

Independent Component Analysis (ICA) is a statistical technique for separating mixed signals into independent non-Gaussian signals. ICA finds statistically independent components in mixed signals by reducing mutual information between outputs. It can be used for signal processing applications like the "cocktail party effect" as well as image processing tasks like separating overlapped or reflected images into their independent sources. ICA uses an unmixing matrix to perform a linear transformation on mixed data to maximize statistical independence between outputs.

Uploaded by

Gabriel Humpire
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 28

Independent Component Analysis

‹ Derek Beaton
Independent Component Analysis

Image, and signal processing solution


to mixed multiple sources and linear
distortion.

Currently in ICA – only linear


distortion can be removed, a NLICA is
being developed.
What is ICA?

‹ ICA is a powerful tool that finds


statistically independent points in mixed
signals.
‹ By finding these points – we reduce
mutual data – working with sets we
know we need
‹ Entirely decided by probability – no
judgment made by user for points.
Some examples…

‹ Examples of where and how ICA is


currently being used:
– Signal Processing: “Cocktail Party Effect”
– Neurophysiological studies: Brain
Waves/EEG data
– Image Processing (obviously): Multiple
overlapped images or reflections
Cocktail Party Effect
The Outcome
So Essentially…

Eight Happy Dudes equals eight happy files


So where is the Image
Processing?
‹ Here it is…
‹ The top two images are
taken with a camera
with a different light
setting in a room – A
manikin and a painting.
The bottom images are
the separated sources
(in this case, we have
an instance of
reflection removal)
A Test of Mixed Images!

‹ Me and Supermodel
Rachel Perry…
TOGETHER AT
LAST!

‹ I’ve input these


images twice each
for this pass… so
what will happen?
Oh no – we’ve separated!
A Reflection Example

‹ This shows
a feature of
ICA –
extracting an
image from
an image
made from
reflection.
Still Reflecting…
Remove the Reflections!
More Fishy-ness!
To the difficult stuff!

‹ So what does it do, and how does it do


it?
– Find natural coordinate system for
empirical data.
– ICA uses linear transformation (an
unmixing matrix)
– Data are maximally, statistically
independent.
Math Junk!

‹ Ck = denotes the class K; Ck is assumed


to be known in advance.
‹ Ok = unknown parameters for each x of
Class K. a.k.a Component Densities.
‹ Assume Component Densities are non-
Gaussian.
Equations

‹ This applies with the assumption that we have


an equal number of sources, and equal
number of linear combinations…
‹ Mixture Density:
– Our data X = {x1,… , xt}
– P(xt|O) = ∑ p(xt | Ck, Ok) p(Ck)
 Also: P(xt|O) = P(xt, O)/P(O) where P(xt, O) = P(xt) *
P(O)
– Data Components cannot be GAUSSIAN, and are
described as xt = AkSk + bk
 A is M x N matrix; b is a bias vector (for finding individual “components”
Equations

‹ Learning Algorithm… Shortened…


– This computes the log likelihood of data for each class:
 log p(xt | Ck, Ok) = log P(Sk)–log (det |Ak |)
– Then compute probability on each class.
– Adapt Ak and the Bias bk for every Ck – Through Gradient
Ascent.
– Gradient is assumed through summation of multiple data
points – or through the ICA algorithm itself.
– Gradient - change in the value of a quantity with change in a
given variable and especially per unit on a linear scale
Summing things up.

‹ We need to rotate our axis of our


input matrix (at the time). This will
minimize our Gaussianity. ICA is
able to recover the original sources
which are statistically independent.
This is provided through the Central
Limit Theorem.
‹ CLT states: any linear mixture of 2
independent random variables is
more Gaussian than the original
variables.
‹ In MatLab to find the Gaussianity –
use the kurtosis() function found in
the Statistical Toolbox.
‹ Did you know… You can prove the
Central Limit Theorem with inverse
Fourier transformations?
Infomax Simplified

‹ Image some Matrices here…


– Mapping: f : x Æ y
– a = Wx;
– Where “W” is a linear transformation
– y = g(a) a function based on m sources or inputs;
– Where ym = gm (am);
– And where g is a bounded nonlinearity applied to each output (am);
– We receive each one of our (am) from the a = Wx formula, where x is each
individual input, transformed.
– Our newly mapped data is mutual data, data that needs to be minimized:
– I[x;y] = H[x] + H[y] – H[x, y]
– = H[y] – H[x|y]
– The entropy (measured randomness) found in all the outputs, H[y], can be
written in terms of the mutual data that has been discovered between
individual outputs,
– I[y]:
– I[x;y] = ∑Mm = 1 H[ym] – I[y] – H[x|y].
Some Semi-Visuals…

‹ X is the original data


‹ S is the source of the activity
‹ To retrieve our W (weight ) matrix – we multiply each component (vector) in S by our X
matrix, and store that data as a component matrix. If we decide that we have a level of
distortion – we subtract our component matrix from our original data.
‹ Components of S
‹ [x y z]
‹ [a b c]
‹ [l m n]
‹ Activity in W
‹ [1]
‹ [2]
‹ [3]
‹ Resulting matrix (vector):
‹ [(x*1)+(y*1)+(z*1)]
‹ [(a*2)+(b*2)+(c*2)]
‹ [(l*3)+(m*3)+(n*3)]
‹ Repeating this process for all Components of S and all Activities in W, will result in a matrix
of equal size to X (original data) and subtracting it removes the distortion
Fun Stuff!

‹ So those were the essentials in the


finding of independent components.
Now that I’ve lost your attention…
‹ These functions/equations are based in
the InfoMax algorithm – a Neural
Network based ICA algorithm –
combining the best of both worlds.
ICA – The 8th Wonder of the
World.
‹ ICA has been presented here as a reflection
removal tool, or even a multiple image
detection then extraction tool. However… it
doesn’t end there…
– Image De-noising
– Fill in the blanks (this is actually implemented
using the De-Noising technique – missing pixels
are consider Noise.)
Just for Fun(ny) originals…
Just for Fun(ny)… mixed
A Closer Look…

Here they are! Please note – that a ~ implies a complement, or


negative image. And that also ICA will produce different results
almost every time. This was the 5th CLEAN run of ICA (SANG).
Questions/Comments/Complaints

You might also like