Gene and Sample Clustering

Clustering genes and samples

Clustering genes and samples is a common analysis method in genomics, typically used to identify
patterns in gene expression data. This approach helps group genes with similar expression profiles
across various samples, or vice versa. Here’s a breakdown of the steps and techniques involved:

1. Data Preparation

Normalize the data to make expression levels comparable across samples and genes.
Filter genes or samples if necessary to remove noise or irrelevant entries.

2. Choose a Distance Metric

Common metrics include Euclidean distance, Manhattan distance, or correlation-based

distances (e.g., Pearson or Spearman correlation).
The choice of distance metric can impact how clusters are formed, especially for high-dimensional
gene expression data.

3. Apply Clustering Algorithm

Hierarchical Clustering: Produces a tree-like structure (dendrogram) showing relationships

among genes or samples.
K-means Clustering: Groups genes or samples into k clusters based on the chosen distance
metric. It's commonly used for larger datasets.
Density-based methods (like DBSCAN): Useful if there are clusters of varying densities and
shapes.

4. Visualize Clustering

Heatmaps: Often used to show the clustered data. Genes (rows) and samples (columns) are
reordered based on clustering, highlighting co-expressed gene groups or similar samples.
Dendrograms: Generated from hierarchical clustering to visualize how genes or samples are
merged at different similarity levels.

5. Interpretation and Validation

Analyze clusters to identify meaningful biological patterns, such as groups of co-expressed genes
or samples with similar phenotypes.
Biological Enrichment Analysis: Run analyses (e.g., Gene Ontology or pathway enrichment) to
identify any biological processes enriched in gene clusters.
Cross-validation techniques or bootstrapping can assess the robustness of clusters.

Printed using Save ChatGPT as PDF, powered by PDFCrowd HTML to PDF API. 1/5
Tools and Libraries

R: `heatmap`, `pheatmap`, `cluster`, and `ComplexHeatmap` packages.

Python: `scikit-learn`, `SciPy`, and `seaborn` for clustering and visualization.

Would you like more details on a specific clustering algorithm or visualization method?

eucledean distance

Euclidean distance is one of the most commonly used distance metrics in clustering and other machine
learning tasks. It calculates the straight-line distance between two points in Euclidean space, making it
intuitive for measuring similarity or dissimilarity between two data points (such as genes or samples in a
dataset).

Formula

For two points p = (p1 , p2 , … , pn ) and q = (q1 , q2 , … , qn ), the Euclidean distance d(p, q) is given by:
​ ​ ​ ​ ​ ​

d(p, q) = (p1 − q1 )2 + (p2 − q2 )2 + … + (pn − qn )2

​ ​ ​ ​ ​ ​ ​

Application in Clustering

1. Interpretation: A smaller Euclidean distance between two points indicates higher similarity, while
a larger distance suggests greater dissimilarity.
2. Hierarchical Clustering: Euclidean distance is often used with hierarchical clustering to measure
the distance between individual data points, which are then used to calculate linkage distances
between clusters.
3. K-means Clustering: Euclidean distance is commonly used to assign points to the nearest cluster
center. The algorithm iteratively minimizes the sum of squared Euclidean distances between points
and their assigned cluster centers.

Properties

Sensitive to scale: Because it’s based on absolute distances, the scale of each feature can
significantly impact the distance. It's often necessary to normalize or standardize data before
applying Euclidean distance.
High-dimensional data: In high-dimensional spaces, Euclidean distances between points become
more similar, leading to the “curse of dimensionality.” Dimensionality reduction methods, such as
PCA, are often applied to alleviate this.

Example Calculation

To calculate Euclidean distance between two points (such as expression levels of two genes across
samples) with values, say:

p = (2, 3, 4) and q = (5, 6, 8)

Printed using Save ChatGPT as PDF, powered by PDFCrowd HTML to PDF API. 2/5
Then,

d(p, q) = (2 − 5)2 + (3 − 6)2 + (4 − 8)2 = ​ (−3)2 + (−3)2 + (−4)2 = ​ 9 + 9 + 16 =​ 34 ≈ ​

When to Use Euclidean Distance

Euclidean distance is best used when:

Features have similar scales or are normalized.

The data is continuous, as it doesn’t work well with categorical data.
The data has relatively low dimensionality or after applying dimensionality reduction.

If you’re looking into implementing this in code or using it in a specific clustering method, I can provide
more detailed steps for that too!

dendogram

A dendrogram is a tree-like diagram that is commonly used to represent the hierarchical relationships
between objects, such as genes or samples, in a dataset. It is a visualization of hierarchical clustering,
which organizes data into nested clusters based on similarity.

How a Dendrogram Works

1. Data Pairing: Hierarchical clustering algorithms start by treating each data point as its own cluster.
2. Calculate Distance: At each step, the algorithm calculates the distance between each pair of
clusters using a specified distance metric (such as Euclidean distance) and a linkage criterion.
3. Merge Clusters: The two closest clusters are merged to form a new cluster.
4. Repeat Steps: This process continues until all points are merged into a single cluster.

The result is a nested clustering structure that can be represented as a dendrogram, where:

Leaves represent individual data points (e.g., genes or samples).

Branches represent clusters that group these points.
Height of each node (where branches merge) indicates the distance (or dissimilarity) between the
merged clusters. Higher nodes mean more dissimilar clusters.

Linkage Criteria

The choice of linkage criterion affects how clusters are merged and how the dendrogram is shaped.
Common linkage methods include:

Single Linkage: Distance between the closest points of two clusters.

Complete Linkage: Distance between the farthest points of two clusters.
Average Linkage: Average distance between all points in two clusters.
Ward's Linkage: Minimizes the increase in total within-cluster variance after merging.

Uses of Dendrograms

Printed using Save ChatGPT as PDF, powered by PDFCrowd HTML to PDF API. 3/5
 Identifying Clusters: By "cutting" the dendrogram at a certain height, you can define clusters
based on the similarity threshold. Everything below the cut forms a cluster.
Visualizing Gene Expression: In gene expression data, dendrograms can show groups of co-
expressed genes or similar samples, helping identify biologically relevant patterns.
Understanding Data Structure: Dendrograms give insights into the natural grouping and
hierarchical structure of the data.

Dendrogram in Practice

1. In Python: You can use `scipy.cluster.hierarchy.dendrogram` along with `linkage` to perform

hierarchical clustering and create a dendrogram.

python

from scipy.cluster.hierarchy import dendrogram, linkage

import matplotlib.pyplot as plt

# Sample data
data = [[1, 2], [2, 3], [3, 4], [5, 8], [8, 8]]

# Perform hierarchical clustering

Z = linkage(data, method='ward') # or use 'single', 'complete', etc.

# Plot the dendrogram

plt.figure(figsize=(10, 7))
dendrogram(Z)
plt.show()

2. In R: Use the `hclust` function with `dist` for distance calculation, and `plot` for visualization.

Interpretation Tips

Cutting the Dendrogram: Setting a threshold and cutting the dendrogram at a certain height lets
you define clusters. Everything below that cut forms clusters of similar items.
Cluster Hierarchies: The hierarchical nature lets you see both small and large clusters. Smaller
branches represent tight clusters, while larger branches represent more generalized groupings.

Let me know if you'd like specific help with code, or interpreting results!

why z test and t test sample size is greater than 30 and less than 30 respectively

The Z-test and t-test are both statistical methods used to test hypotheses about population means, but
they differ in terms of the assumptions they make about the data, particularly in relation to sample size
and population variance.

1. Z-Test: Sample Size Greater Than 30

The Z-test is generally used when the sample size is large (typically n > 30), based on the following
assumptions:

Printed using Save ChatGPT as PDF, powered by PDFCrowd HTML to PDF API. 4/5
 Normality: According to the Central Limit Theorem, the distribution of sample means
approaches a normal distribution as the sample size grows, even if the data itself is not normally
distributed. For sample sizes above 30, this approximation is usually sufficient.
Population Standard Deviation Known: The Z-test assumes that the population standard
deviation (σ ) is known or can be accurately estimated. This is practical with larger samples, where
the sample standard deviation can approximate the population standard deviation well.

With larger sample sizes, the Z-test becomes a powerful test due to these assumptions. The large
sample size reduces the influence of individual data points, creating a robust estimate of the population
mean.

2. T-Test: Sample Size Less Than 30

The t-test is typically used when the sample size is small (n < 30) because of these reasons:

Unknown Population Standard Deviation: In smaller samples, it’s less likely that the population
standard deviation is known. Instead, the t-test uses the sample standard deviation (s) to estimate
the population standard deviation, introducing more variability.
T-Distribution: For smaller samples, the sample means follow a t-distribution rather than a
normal distribution, which has "fatter tails" (i.e., it accounts for the increased variability in the
sample standard deviation). The t-distribution converges to a normal distribution as the sample
size increases, making the t-test appropriate for smaller samples.

Summary of When to Use Each Test

Z-Test: Use if n > 30 and the population standard deviation (σ ) is known or can be accurately
estimated.
T-Test: Use if n < 30 or the population standard deviation is unknown and needs to be estimated
from the sample.

In cases where the sample size is large, the difference between the Z-test and t-test becomes negligible,
and both yield similar results because the t-distribution approximates the normal distribution as n
grows. This is why the Z-test is preferred for larger samples where normality can be assumed.

Printed using Save ChatGPT as PDF, powered by PDFCrowd HTML to PDF API. 5/5