14.

1 Sparse Matrix Basics

Consider a graph containing information of cities of India and their direct connectivity. Cities
are shown as vertices and direct connectivity through edges. Let us assume that there are a total
of 1000 vertices and 2000 direct connections. If the graph is represented by Adjacency Matrix,
total entries in the matrix will be 1000x1000 = 100000. Out of these 2000 entries will be 1 and
rest (99800) will be zero. The entries which are of our use is only 2000, rest of the entries are
useless.

A sparse matrix or sparse array is a matrix in which most of the elements are zero. The number
of zero-valued elements divided by the total number of elements (e.g., m × n for an m × n matrix)
is called the sparsity of the matrix. Using those definitions, a matrix will be sparse when its
sparsity is greater than 0.5.

For the above 6x6 matrix, number of zero elements are equal to 25. Sparsity for this matrix is
equal to 25/36 = 0.6944

When storing and manipulating sparse matrices on a computer, it is beneficial and often
necessary to use specialized algorithms and data structures that take advantage of the sparse
structure of the matrix. Operations using standard dense-matrix structures and algorithms are
slow and inefficient when applied to large sparse matrices as processing and memory are wasted
on the zeroes. Sparse data is by nature more easily compressed and thus requires significantly
less storage. Some very large sparse matrices are infeasible to manipulate using standard dense-
matrix algorithms.

14.2. Types of Sparse Matrix

Sparse matrix can be categorized into two
- Sparse Matrix with pattern
- Sparse Matrix without a pattern
14.2.1. Sparse Matrix with Pattern
The following matrices exhibit some pattern of non-zero elements
 Identity Matrix:- where only diagonal elements store 1. The rest of the elements is zero.
 Diagonal Matrix:- where only diagonal elements store non zero value. Rest of the
elements is zero.
 Lower Triangular Matrix:- In this kind of matrix the diagonal elements and elements
below contain non zero elements. The rest of the elements is zero.
 Upper Triangular Matrix:- In this kind of matrix the diagonal elements and elements
above contain non-zero elements. The rest of the elements is zero.
 Tridiagonal Matrix:- where only diagonal elements store 1. Rest of the elements is zero.

Figure: Diagonal Sparse Matrix Triangular Sparse Matrix Tridiagonal sparse Matrix

14.2.2 Sparse Matrix without Pattern

The matrices of such kind do not have any pattern of non-zero or zero elements e.g., the
Adjacency matrix of Graph.

Figure: Showing the Adjacency matrix of the graph that has several zeros
14.3 Representing sparse matrix to save memory

• A Matrix of size N x N may be stored as a N x N Array. This would require ϴ(N2) space.
• Given sparse matrices may be mapped into a one-dimensional Array by storing only
non-zero elements.
• A mapping function can locate a non-zero element of matrix in the 1-D array. This
function should give the location in constant time.
• The space needed by the 1D array would be less than the space required by a N x N 2-D
Array

14.3.1 Identity Matrix:

ith row and ith column element of matrix can be mapped to ith index in 1-D Array (assuming the
indexes starting with 1)

e.g. Index of M[1][1] in the one dimensional array will be 1

Index of M[2][2] in the one dimensional array will be 2
Index of M[3][3] in the one dimensional array will be 3, and so on so forth

14.3.2 Diagonal Sparse Matrix:

Consider a sparse diagonal matrix given below:
ith row and ith column element of matrix can be mapped to ith index in 1-D Array (assuming the
indexes starting with 1).
To save memory, we can use a one-dimensional array. Array index for <i, i> element of the
Matrix will be stored at i index in the one-dimensional array.
e.g. Index of M[1][1] in the one dimensional array will be 1
Index of M[2][2] in the one dimensional array will be 2
Index of M[3][3] in the one dimensional array will be 3, and so on so forth

14.4 Storage of Sparse Matrices with No Pattern:

Sparse matrices with no pattern can be stored with two methods:

- Vector Representation and
- Linked list Representation

14.4.1 Vector Representation:

In computer programming, a vector is either a pointer or an array with only one dimension.
However, in mathematics, a vector is a quantity with both a magnitude and direction. Vector
representation stores the data item in the form of a triplet <Row, Column, Data>. For the storage
of elements, information about their row & column number is a must as it will help us carry out
computations. The example given below shows the representation of a sparse matrix in the form
of an array of structures.

Structure can be defined as:

struct sparse
{
int row;
int col;
int value;
};
struct sparse s[10];
14.4.2 Linked List Representation:
A linked list is a data structure that consists of nodes. Each node contains at least two fields. One
contains the data and the other one stores the address of the next node.

For Linked list representation of sparse matrix, the node is used to store the sparse matrix
elements. The node is supposed to contain 4 fields.

Example given below shows the linked list representation of the given sparse matrix. Here, row
major order representation of Matrix is considered.
14.5 Addition of Two Sparse Matrices

Suppose two sparse matrices are given as the linked lists. For addition to be performed, row
and column numbers of the matrix should be the same for the current node. The logic for the
addition is given in the Algorithm shown below

ALGORITHM SparseMatrixAddition (Spase1, Sparse2)

BEGIN:
p=Sparse1
q=Sparse2
Sparse3=NULL
WHILE p!=NULL AND q!=NULL DO
IF p→row = = q→row THEN
IF p→column = = q→ column THEN
InsBeg(Sparse3, p→row, p→column, p→element+q→element)
p=p→Next
q=q→Next

ELSE
IF p→column< q→ column THEN
InsBeg(Sparse3, p→row, p→column, p→element)
p=p→Next
ELSE
InsBeg(Sparse3, q→row, q→column, q→element)
q=p→Next
ELSE

IF p→row<q→row THEN
InsBeg(Sparse3, p→row, p→column, p→element)
p=p→Next
ELSE
InsBeg(Sparse3, q→row, q→column, q→element)
q=q→Next
WHILE p!=NULL DO

InsBeg(Sparse3, p→row, p→column, p→element)

p=p→Next
 WHILE q!=NULL DO
InsBeg(Sparse3, q→row, q→column, q→element)
p=p→Next

Reverse(Sparse3)
RETURN Sparse3
END;

The algorithm makes the comparison to find the same row and same column elements. The two
are added and stored as a node in the answer linked list. Other elements from original linked lists
are added in the answer linked list assuming the equivalent element in the other linked list is 0.
Since the elements are added in the opposite order of row and column numberin the final linked
list, it needs to be reversed before producing the answer.

For performing the addition of above two matrices, both the matrices are represented in the
form of linked lists. Given linked list are:
 Performing the Addition according to the Algorithm:

Step 1:

Matrix A

Matrix B

A+B

Step 2:

Matrix A

Matrix B

A+B

Step 3:

Matrix A

Matrix B

A+B

Step 4:

Matrix A

Matrix B

A+B
 Step 5:

Matrix A

Matrix B

A+B

Step 6:

Matrix A

Matrix B

A+B

Step 7:

Matrix A

Matrix B

A+B

Step 8:

Matrix A

Matrix B

A+B
 Step 9:

Matrix A

Matrix B

A+B

Step 10:

Matrix A

Matrix B

A+B

Step 11:

Matrix A

Matrix B

A+B

Time Complexity: Considering M and N the number of non-zero elements in the two sparse
matrices, the final linked list will be formed with an effort of O(M+N). Reverse operation (In place)
also requires just 1 traversal; hence the O(M+N) effort is required for that as well. Total effort
hence is O(M+N) + O(M+N) i.e. O(M+N).
Space Complexity: Considering M and N the number of non-zero elements in the two sparse
matrices, the final linked list will the size M+N. Few more variables in the algorithm make that
constant space. Hence the total space required is O(M+N).
14.6 Transpose of a Sparse Matrices

Transpose of a sparse matrix requires the exchange of row numbers with the column no for all
the elements. Since the order of row and column, which earlier was arranged as row-major order
fashion, has got disturbed needs to be corrected. A sorting should be performed on answer linked
list in the form of row and column no. The diagrams given below shows the working of finding
the transpose of the given matrix.

ALGORITHM SparseMatrixTanspose (Sparse)

BEGIN:
p=Sparse1
Sparse2=NULL
WHILE p!=NULL DO
Exchange (p→row, p→column)
p=p→Next
Sort(Sparse2)
RETURN Sparse2
END;
Finding the transpose requires the swapping of row and the column number of each of the data
item. Since the order of linked list (Row major) will change, some sorting us required to regain
the row-major order sequence.

Time Complexity: Traversal is required for performing the exchanges. Each exchange requires
3 statements. Effort upto exchanges are O(N). Sorting requires additional O(NLogN) effort. Total
effort hence is O(N) + O(NLogN) = O(NLogN)
Space Complexity: O(1) as only a few variables are required for traversal, exchange and sorting.