Independent Component Analysis - ML
Last Updated :
11 Jul, 2025
Independent Component Analysis (ICA) is a technique used to separate mixed signals into their independent, non-Gaussian components. Its aim to find a linear transformation of data that maximizes statistical independence among the components. ICA is widely used in fields like audio, image processing and biomedical signal analysis to isolate distinct sources from mixed signals.
Statistical Independence Concept
Statistical independence refers to the idea that two random variables: X and Y are independent if knowing one does not affect the probability of the other. Mathematically this means the joint probability of X and Y is equal to the product of their individual probabilities.
P(X and Y) = P(X) * P(Y) or P(X \cap Y) = P(X) * P(Y)
Assumptions in ICA
ICA operates under two key assumptions:
- The source signals are statistically independent of each other.
- The source signals have non-Gaussian distributions.
These assumptions allow ICA to effectively separate mixed signals into independent components, a task that traditional methods like PCA cannot achieve
Mathematical Representation of ICA
The observed random vector is X= (x_1 , ..., x_m )^T representing the observed data with m components. The hidden components are represented by the random vector S = (s_{1} , ..., s_{n})^T where n is the number of hidden sources.
The observed data X is transformed into hidden components S using a linear static transformation representation by the matrix W.
S = WX
Here, W = transformation matrix.
The goal is to transform the observed data X in a way that the resulting hidden components are independent. The independence is measured by some function F(s_1 , ..., s_n). The task is to find the optimal transformation matrix W that maximizes the independence of hidden components.
Cocktail Party Problem in ICA
To better understand how Independent Component Analysis (ICA) works let’s look at a classic example known as the Cocktail Party Problem

Here there is a party going into a room full of people. There is 'n' number of speakers in that room and they are speaking simultaneously at the party. In the same room, there are also 'n' microphones placed at different distances from the speakers which are recording 'n' speakers' voice signals. Hence the number of speakers is equal to the number of microphones in the room.
Now using these microphones' recordings, we want to separate all the 'n' speakers voice signals in the room given that each microphone recorded the voice signals coming from each speaker of different intensity due to the difference in distances between them.
Decomposing the mixed signal of each microphone's recording into an independent source's speech signal can be done by using the machine learning technique independent component analysis.
[ X_1, X_2, ...., X_n ] => [ Y_1, Y_2, ...., Y_n ]
where X1, X2, ..., Xn are the original signals present in the mixed signal and Y1, Y2, ..., Yn are the new features and are independent components that are independent of each other.
Implementing ICA in Python
FastICA is a specific implementation of the Independent Component Analysis (ICA) algorithm that is designed for efficiency and speed.
Step 1: Import necessary libraries
First we will import numpy, sklearn, FastICA and matplotlib.
Python
import numpy as np
from sklearn.decomposition import FastICA
import matplotlib.pyplot as plt
Step 2: Generate Random Data and Mix the Signals
In this step we create three separate signals: a sine wave, a square wave and random noise. These represent different types of real-world signals. We use NumPy to generate 200 time points between 0 and 8.
- signal_1: A sine wave, e.g., like a tuning fork tone.
- signal_2: A square wave, e.g., like a digital signal (on/off).
- signal_3: A random noise signal using the Laplace distribution which has sharper peaks than normal distribution.
- np.c_[]: Combine all three 1D signals into a single 2D array.
0.2 is
the standard deviation of the noise
Python
np.random.seed(42)
samples = 200
time = np.linspace(0, 8, samples)
signal_1 = np.sin(2 * time)
signal_2 = np.sign(np.sin(3 * time))
signal_3 = np.random.laplace(size=samples)
S = np.c_[signal_1, signal_2, signal_3]
S += 0.2 * np.random.normal(size=S.shape)
A = np.array([[1, 1, 1], [0.5, 2, 1], [1.5, 1, 2]])
X = S.dot(A.T)
Step 3: Apply ICA to unmix the signals
In this step we apply Independent Component Analysis using the FastICA class from Scikit-learn. We first create an instance of FastICA and set the number of independent components to 3 matching the number of original signals.
Python
ica = FastICA(n_components=3)
S_ = ica.fit_transform(X) # Estimated sources
Step 4: Visualize the signals
In this step we use Matplotlib to plot and compare the original sources, mixed signals and the signals recovered using ICA.
We create three subplots:
- First shows the original synthetic signals
- Second displays the observed mixed signals
- Third shows the estimated independent sources obtained from ICA
Python
# Plot the results
plt.figure(figsize=(8, 6))
plt.subplot(3, 1, 1)
plt.title('Original Sources')
plt.plot(S)
plt.subplot(3, 1, 2)
plt.title('Observed Signals')
plt.plot(X)
plt.subplot(3, 1, 3)
plt.title('Estimated Sources (FastICA)')
plt.plot(S_)
plt.tight_layout()
plt.show()
Output:

Difference between PCA and ICA
Both the techniques are used in signal processing and dimensionality reduction but they have different goals.
Principal Component Analysis | Independent Component Analysis |
---|
It reduces the dimensions to avoid the problem of overfitting. | It decomposes the mixed signal into its independent sources signals. |
It deals with the Principal Components. | It deals with the Independent Components. |
It focuses on maximizing the variance. | It doesn't focus on the issue of variance among the data points. |
It focuses on the mutual orthogonality property of the principal components. | It doesn't focus on the mutual orthogonality of the components. |
It doesn't focus on the mutual independence of the components. | It focuses on the mutual independence of the components. |
Advantages of Independent Component Analysis (ICA):
- ICA is a powerful tool for separating mixed signals into their independent components. This is useful in a variety of applications such as signal processing, image analysis and data compression.
- It is a non-parametric approach which means that it does not require assumptions about the underlying probability distribution of the data.
- It is an unsupervised learning technique which means that it can be applied to data without the need for labeled examples. This makes it useful in situations where labeled data is not available.
- This can be used for feature extraction which means that it can identify important features in the data that can be used for other tasks, such as classification.
Disadvantages of Independent Component Analysis (ICA):
- It assumes that the underlying sources are non-Gaussian, which may not always be true. If the underlying sources are Gaussian ICA may not be effective.
- ICA assumes that the sources are mixed linearly which may not always be the case. If the sources are mixed nonlinearly ICA may not be effective.
- This can be computationally expensive especially for large datasets which make it difficult to apply ICA to real-world problems.
Do Check this article for better understanding:
Similar Reads
Machine Learning Tutorial Machine learning is a branch of Artificial Intelligence that focuses on developing models and algorithms that let computers learn from data without being explicitly programmed for every task. In simple words, ML teaches the systems to think and understand like humans by learning from the data.Do you
5 min read
Introduction to Machine Learning
Python for Machine Learning
Machine Learning with Python TutorialPython language is widely used in Machine Learning because it provides libraries like NumPy, Pandas, Scikit-learn, TensorFlow, and Keras. These libraries offer tools and functions essential for data manipulation, analysis, and building machine learning models. It is well-known for its readability an
5 min read
Pandas TutorialPandas is an open-source software library designed for data manipulation and analysis. It provides data structures like series and DataFrames to easily clean, transform and analyze large datasets and integrates with other Python libraries, such as NumPy and Matplotlib. It offers functions for data t
6 min read
NumPy Tutorial - Python LibraryNumPy (short for Numerical Python ) is one of the most fundamental libraries in Python for scientific computing. It provides support for large, multi-dimensional arrays and matrices along with a collection of mathematical functions to operate on arrays.At its core it introduces the ndarray (n-dimens
3 min read
Scikit Learn TutorialScikit-learn (also known as sklearn) is a widely-used open-source Python library for machine learning. It builds on other scientific libraries like NumPy, SciPy and Matplotlib to provide efficient tools for predictive data analysis and data mining.It offers a consistent and simple interface for a ra
3 min read
ML | Data Preprocessing in PythonData preprocessing is a important step in the data science transforming raw data into a clean structured format for analysis. It involves tasks like handling missing values, normalizing data and encoding variables. Mastering preprocessing in Python ensures reliable insights for accurate predictions
6 min read
EDA - Exploratory Data Analysis in PythonExploratory Data Analysis (EDA) is a important step in data analysis which focuses on understanding patterns, trends and relationships through statistical tools and visualizations. Python offers various libraries like pandas, numPy, matplotlib, seaborn and plotly which enables effective exploration
6 min read
Feature Engineering
Supervised Learning
Unsupervised Learning
Model Evaluation and Tuning
Advance Machine Learning Technique
Machine Learning Practice