Sparse Matrix 1

What is a sparse matrix?

A sparse matrix is a special case of a matrix in which the number of zero elements is
much higher than the number of non-zero elements. As a rule of thumb, if 2/3 of the
total elements in a matrix are zeros, it can be called a sparse matrix. Using a sparse
matrix representation — where only the non-zero values are stored — the space
used for representing data and the time for scanning the matrix are reduced
significantly.

Many applications in data science and machine learning involve sparse matrices,
such as:

• Natural Language Processing: The occurrence of words in documents can be

represented in a sparse matrix. The words in a document are only a small subset of
words in a language. If we have a row for every document and a column for every
word, then storing the number of word occurrences in a document has a high
percentage of zeros in every column.
• Recommendation Systems: A sparse matrix can be employed to represent whether
any particular user has watched any movie. See our Locality Sensitive Hashing
(LSH) article for an example.
• Market Basket Analysis: Since the number of purchased items is tiny compared to
the number of non-purchased items, a sparse matrix is used to represent all
products and customers.
 Sparse Matrix 2
Numerical example 1

Let's take the example of a movie recommendation system. There are millions of
users and thousands of movies, so it's not possible for users to watch and rate all
movies. This data can be represented as a matrix where the rows are the users, and
the columns are the movies.

Most of the matrix elements will be empty, where the missing values will be
replaced with zeros. Since a small percentage of the matrix has non-zero values, this
matrix can be considered a sparse matrix. A small portion of the data is given below:

Movie1 Movie2 Movie3 Movie4 Movie5 Movie6 Movie7

User1 0 0 0 3 0 0 4
User2 0 5 0 0 0 0 0
User3 0 0 5 0 0 4 0
User4 4 0 0 0 0 0 1
User5 0 2 0 0 3 0 0
The sparsity of this matrix can be calculated by obtaining the ratio of zero elements
to total elements. For this example, sparsity is calculated as:

sparsity=nzerosntotal=2635=0.742

It can be seen that the number of zeros in a sparse matrix is very high. Representing
all zero values in a matrix like this would result in high memory usage, so in
practice, only non-zero values of the sparse matrix are stored.

Numerical example 2

Another example would be to use a matrix to represent the occurrence of words in

documents. The term-document matrix dimension will be n×m, where n is the
number of documents and m is the number of words in the language model. As a
result, most of the matrix elements will be zero since only non-zero values are
important for data analysis. In addition to a large amount of space used, there will
be a computational time problem because all elements will be scanned to access
non-zero elements. This process yields a computational complexity problem.

To overcome these problems, we can use different data structures to represent a

sparse matrix. One common representation format for a sparse matrix is
a Coordinate list (COO), which uses three-element tuples to store non-zero values'
 Sparse Matrix 3
coordinates in a matrix. For example, the following table can be constructed to
represent a sparse term-document matrix:
Row Column Value
0 3 3
0 6 4
1 1 5
2 2 5
2 5 4
3 0 4
3 6 1
4 1 2
4 4 3
In this table, indices of rows and columns of non-zero values are stored in a sparse
representation. Let k be the number of non-zero elements in a matrix of size n×m,
then the proportion of the space saved by sparse matrix representation can simply
be calculated as follows:
p=1−3knm

The space gained by a sparse matrix representation is directly proportional to the

sparsity value.
 Sparse Matrix 4
Generating Compact Sparse Matrix
From Sparse Matrix

To generate a compact form of a sparse matrix from a normal sparse matrix in C, you can store
only the non-zero elements along with their row and column positions. This is more memory-
efficient for sparse matrices, where most of the elements are zero.

Steps:

1. Normal Sparse Matrix: A sparse matrix that is represented by a 2D array, but you want
to store only the non-zero elements.
2. Compact Form: A more efficient representation, where only the non-zero elements and
their row/column positions are stored.

Example:

We will convert the normal sparse matrix (2D array) into a compact form, which will only store
the non-zero values along with their row and column indices.

C Code:
#include <stdio.h>

#define MAX_SIZE 100

// Structure to represent a term in the sparse matrix

typedef struct {
int row;
int col;
int value;
} term;

// Function to generate compact form of the sparse matrix

void generateCompactForm(int matrix[MAX_SIZE][MAX_SIZE], term
compact[MAX_SIZE * MAX_SIZE], int rows, int cols) {
int k = 0; // Index to store the current non-zero element in compact
form
for (int i = 0; i < rows; i++) {
for (int j = 0; j < cols; j++) {
if (matrix[i][j] != 0) {
k++;
compact[k].row = i; // Row index of non-zero element
compact[k].col = j; // Column index of non-zero element
compact[k].value = matrix[i][j]; // Value of non-zero
element
// Move to the next non-zero element
}
}
}
}
compact[0].row = rows; // Rows
compact[0].col = cols; // Column index of non-zero element
compact[0].value = k
 Sparse Matrix 5
// Function to print the compact form of the sparse matrix
void printCompactForm(term compact[MAX_SIZE * MAX_SIZE], int n) {
printf("Compact Form (Row, Column, Value):\n");
for (int i = 0; i < n; i++) {
printf("Row: %d, Col: %d, Value: %d\n", compact[i].row,
compact[i].col, compact[i].value);
}
}

int main() {
int matrix[MAX_SIZE][MAX_SIZE] = {
{0, 0, 3, 0, 4},
{5, 0, 0, 0, 0},
{0, 0, 0, 0, 0},
{0, 0, 0, 0, 6}
};

int rows = 4, cols = 5;

// Create an array to store the compact form of the matrix

term compact[MAX_SIZE * MAX_SIZE];

// Generate the compact form

generateCompactForm(matrix, compact, rows, cols);

// Count the number of non-zero elements in the matrix

int nonZeroCount = 0;
for (int i = 0; i < rows; i++) {
for (int j = 0; j < cols; j++) {
if (matrix[i][j] != 0) {
nonZeroCount++;
}
}
}

// Print the compact form

printCompactForm(compact, nonZeroCount);

return 0;
}

Explanation:

1. Matrix Representation:
o The normal sparse matrix is represented as a 2D array
matrix[MAX_SIZE][MAX_SIZE].
o The compact form is represented by an array of term structures, each holding the
row, column, and value of a non-zero element.
2. generateCompactForm Function:
o Loops through the entire matrix, checks for non-zero values, and stores them
along with their row and column indices in the compact array.
o Each non-zero element is stored in the compact array in the form (row, column,
value).
3. printCompactForm Function:
o Prints the elements stored in the compact array.
 Sparse Matrix 6
4. Main Function:
o Initializes a 2D array matrix representing a sparse matrix.
o Calls the generateCompactForm function to convert the matrix into a compact
form.
o Counts the number of non-zero elements.
o Prints the compact form.

Example Output:

For the input matrix:

0 0 3 0 4
5 0 0 0 0
0 0 0 0 0
0 0 0 0 6

The compact form output would be:

Compact Form (Row, Column, Value):

Row: 0, Col: 2, Value: 3
Row: 0, Col: 4, Value: 4
Row: 1, Col: 0, Value: 5
Row: 3, Col: 4, Value: 6
 Sparse Matrix 7
Sparse Matrix
Transpose of Compact Sparse Matrix
#include <stdio.h>

typedef struct {
int row;
int col;
int value;
} term;

void transpose(term a[], term b[]) {

int n, i, j, currentb;

// Number of elements in the matrix

n = a[0].value;

// Set rows and columns of the transpose matrix b

b[0].row = a[0].col;
b[0].col = a[0].row;
b[0].value = n; // The number of non-zero elements remains the same in
transpose

// Initialize currentb to 1 (since b[0] is already initialized)

currentb = 1;

// Loop through each column of matrix a

for (i = 0; i < a[0].col; i++) {
// Find elements in the current column of a
for (j = 1; j <= n; j++) {
if (a[j].col == i) {
// Element is in the current column of a, add it to b
b[currentb].row = a[j].col;
b[currentb].col = a[j].row;
b[currentb].value = a[j].value;
currentb++;
}
}
}
}

int main() {
term a[] = {
{5, 5, 4}, // Matrix size (5 rows, 5 columns, 4 non-zero elements)
{0, 1, 3}, // Non-zero element at (0, 1) with value 3
{1, 0, 5}, // Non-zero element at (1, 0) with value 5
{2, 3, 8}, // Non-zero element at (2, 3) with value 8
{3, 4, 6} // Non-zero element at (3, 4) with value 6
};

term b[5]; // To store the transposed matrix

transpose(a, b);

// Print the transposed matrix

printf("Transposed Matrix:\n");
 Sparse Matrix 8
for (int i = 0; i <= b[0].value; i++) {
printf("Row: %d, Column: %d, Value: %d\n", b[i].row, b[i].col,
b[i].value);
}

return 0;
}

Explanation:

1. Matrix Representation: The sparse matrix is represented by an array of term structures,

where each structure holds:
o row: Row index.
o col: Column index.
o value: The non-zero value at that position.
2. Transpose Function:
o The transpose of a matrix swaps the rows and columns, meaning that an element
at position (i, j) in matrix A would be moved to (j, i) in matrix B.
o We traverse through the original matrix (array a[]) and add elements to the
transposed matrix b[], with updated row and column indices.
3. Main Function:
o Initializes a sparse matrix a[] with some non-zero elements.
o Calls the transpose function to generate the transposed matrix b[].
o Prints the transposed matrix.