MARCH 2016 1

Compressed Sparse Row (CSR)

Kiarash Torkian

Abstract—Paper focuses on highlighting the work and research B. Compressed Sparse Row
that was put into comparing operations done on a regular spare
matrix vs. a compressed one. A library of operations were written
A compressed sparse row is one of the many ways of
using the Java language to compare the two and is publicly compressing a matrix by holding onto any value that is nonzero
available. A compressed matrix will benefit from not having to (val), the column index of values (col ind), and a row pointer
explicitly perform operations on zeros which won’t have any effect that represents the start of a new row (row ptr). There includes
on the final value while requiring a substantially less amount of many other famous methods such as compressed column
space to store. storage (CCS) that instead holds onto the row index (row ind)
and a column pointer (col ptr), and coordinate format (COO)
that instead of having a pointer vector, it will simply hold
I. I NTRODUCTION onto the column index (col ind) and the row index (col ind).
When it comes to matrices, having to write methods to
perform operations on them are not that difficult, since they
really just boil down to only two or more nested loops. Now
in terms of memory and performance, this won’t be a huge
problem as long as the matrices themselves are not very
large. But lets not forget that two nested loops is still an O2
algorithm and will quadratically take more memory and be
slower in performance as the matrices grow larger and larger.
Which happens to be very frequent in real life examples.
One thing that we can say for sure is having zeros in a matrix
can make calculating it easier. But not for a computer, if we
don’t make an new algorithm that works around this since it Figure 1: A compressed sparse row can represent a matrix by
will still go over the zero values anyway. So what if we found holding onto only the nonzero values and two more arrays to
a way to compress our matrices in a format that contained only understand their previous positions. This figure is a represen-
the nonzero values without changing the matrix itself? Turns tation of matrix (1)
out we can do this in multiple ways, but in all cases it won’t
work too well unless we have a spare matrix.
Consider nnz is the number of nonzero elements in a matrix
II. BACKGROUND and n is the length of the row ptr vector. The memory
required to store these elements is equivalent to
A. Spare Matrix
Spare matrix is a type of matrix that is mostly filled with 2nnz + n + 1
zero values and is therefore opposite of a dense matrix, which
Where as to store a matrix in all its formality, one would
contains mostly nonzero values. A spare matrix can be vastly
require m ∗ n of memory where n and m represent number
improved in terms of space needed and performance required
of rows and columns respectively. Even though memory is
since majority of its data can be removed and its operations
something we seem to worry about less and less as time goes
skipped. This may not be so true for a dense matrix however
on since it is not as limited as it used to be, it is however,
for having to take the extra time of converting a matrix into
not something we can ignore when working with large data
a compressed format when we would only save on merely a
such as matrices. We can still reach our limits very soon. The
couple of steps. Here we see a simple example of a spare
benefit however, that comes from compressing matrices is not
matrix.
just memory.

1 0 6 5 3 0 0 0
 
III. A LGORITHMS
2 1 0 0 0 3 0 3
0
 0 9 3 0 0 0 0 Now that we have gotten rid of zero values, our operations
0 0 0 4 0 0 0 0 will no longer need to be considered about looping over
A=
0
 (1) zero values that will have no affect in the end result of
0 0 0 5 0 0 0
the operation. In this section we will be going over some
 
0 0 0 0 0 1 0 0

0 0 0 0 4 7 5 0
 of the main implemented functions done for this program.
0 0 0 0 0 0 0 6
MARCH 2016 2

A. Matrix-Vector Multiplication
Lets compare the below two algorithms, where
one operates on a regular matrix while the other,
on a compressed sparse row. Consider the following:

m = N umber of rows

n = N umber of columns

nnz = N umber of nonzero

av = Average number nonzeros per row

Figure 2: Graphic representation of steps for a compressed
matrix, vector multiplication. All the nonzero in the val vector
Algorithm 1 Matrix-Vector Multiplication O(n ∗ m) are only accessed once which makes this algorithm with a
O(nnz) overhead.
Require: A ← Regular Matrix B ← Vector C ← Final Vector
1: for i = 0 to n do
2: for j = 0 to m do
3: c[i] ← c[i] + A[i][j] * b[j]
4: end for B. Matrix Transposition
5: end for
We see a very familiar pattern in regards to Algorithm 3 and
Algorithm 4 compared to our previous two algorithms. For
one, the regular matrix operation once again is a nested loop
Algorithm 2 CSR Matrix-Vector Multiplication (O(m ∗ av)) that grows and shrinks based on the column and row size of the
Require: A ← CSR Matrix B ← Vector C ← Final Vector matrix and the nested loop always starts from zero. Therefore,
1: for i = 0 to m do the algorithm has a performance of O(n ∗ m). The second one,
2: for j = A.row ptr[i] to j <A.row ptr[i+1] do while having considerably more loops, the nested ones behave
3: c[i] ← c[i] + A.val[j] * b[A.col ind[j]] similarly to ones from Algorithm 2 and are therefore not a true
4: end for nested set of loops. When we try to go through the process of
5: end for finding the Big O, we find out that the two nested loops have
identical conditions and one of them can be removed from the
equation.
Looking at Algorithm 1, we can see that it contains two
nested loops. First one loops till N, while the second one
loops till M. Since the nested loop always starts from zero, we O((n ∗ av) + m + (n ∗ av))
can say that this will result in an O(n ∗ m) overhead which
will have a quadratic growth as the matrices grows larger.
Algorithm 2 also has two nested loops but it is actually not a O(2(n ∗ av) + m)
quadratic growth. The first loop only iterates as number of rows
in the matrix while the second loop has a condition using the
O(2(n ∗ av) + m)
row ptr vector to only find out how many nonzero values are in
that row. So it doesn’t iterate from the start. Algorithm 2 has
only a linear growth of O(m ∗ av) or in other words, O(nnz). O((n ∗ av) + m)
It might be hard to see at first, but looking at Figure 2, we
can see that we would end up performing one operation per
each nonzero elements in a row by matching it with the same
index of the vector’s column. In conclusion, we started by Algorithm 3 Matrix Transposition O(n ∗ m)
an algorithm that was O(n ∗ m), which is considered to be Require: A ← CSR Matrix T ← CSR Matrix
moderately slow and turned it into a O(nnz) operation by 1: for i = 0 to n do
modifying our matrices. As long as, this is done on spare 2: for j = 0 to m do
matrices, the performance increase will be huge, whereas 3: temp ← A[i][j]
for dense matrices, not much will change. A matrix for 4: A[i][j] ← A[j][i];
example that contains no zero values whatsoever, will force 5: A[j][i] ← temp
the algorithm to still do O(nnz) which in this case is also 6: end for
7: end for
O(n ∗ m).
MARCH 2016 3

Algorithm 4 CSR Matrix Transposition O(n + m) Algorithm 6 Symmetrical O(n + m + nnz)

Require: A ← CSR Matrix T ← CSR Matrix Require: A ← CSR Matrix T ← CSR Matrix
1: k←0 1: T ← Algorithm 4 on A
2: row count ← new int[] 2: if A != T then
3: col count ← new int[] 3: return false
4: for i = 0 to n do 4: end if
5: for j = row ptr[i] to j <row ptr[i+1] do
6: k ← col ind[j]
7: row count[k] ← row count[k] + 1 IV. C ONCLUSION
8: end for Compressing spare matrices can substantially improve the
9: end for performance and the space requirement of a program. It will
10: for i = 0 to m do
11: T.row ptr[i + 1] ← T.row ptr[i] + row count[i]
allow you to get past quadratic growth algorithms and instead
12: end for have linear ones. Compression techniques, as beneficial as they
13: for i = 0 to n do are, are not always the best options. Using them on dense
14: for j = row ptr[i] to j <row ptr[i+1] do matrices will make no improves in anyway and might not
15: k ← col ind[j] be worth the conversions on other cases. Therefore a small
16: index ← T.row ptr[k] + row count[k] decision making is required before deciding if you want your
17: T.col indx[index] ← i matrices compressed.
18: T.val[index] ← T.val[j]
19: end for
20: end for

C. Symmetrical

Lets also quickly talk about checking whether a matrix is

symmetrical or not. A symmetrical matrix is one where its
inverse is equivalent of itself.

A = AT

A−1 AT = I

The method for a regular matrix would be as usual, two

nested loops where it would try to check if a value in a row,
column is equal to a value in a column, row. If this is not
the case then the matrix is not symmetrical and is therefore
not equal to its transpose. For performing this under our
compressed matrix, we can simply use one of our previously
implemented functions. Here, we can use Algorithm 4 to first
find the transpose of a compressed matrix. The function should
give us back a transposed version of matrix A. We will then
need to make sure the values in both of these matrices are the
same.

Algorithm 5 Symmetrical O(n ∗ m)

Require: A ← CSR Matrix
1: for i = 0 to n do
2: for j = 0 to m do
3: if A[i][j] != A[i][j] then
4: return false
5: end if
6: end for
7: end for
MARCH 2016 4

R EFERENCES
[1] M. Mccourt, B. Smith, and H. Zhang, EFFICIENT SPARSE MATRIX-
MATRIX PRODUCTS USING COLORINGS. [Online]. Available at:
http://www.mcs.anl.gov/papers/p5007-0813 1.pdf
[2] S. Y. Cheung, Working with Sparse Matrices, Work-
ing with Sparse Matrices. [Online]. Available at:
http://www.mathcs.emory.edu/c̃heung/courses/561/syllabus/3-
c/sparse.html
[3] Compressed Sparse Row Format: CSR, Compressed
Sparse Row Format: CSR. [Online]. Available at:
http://www5.in.tum.de/lehre/vorlesungen/parnum/ws10/parnum 6.pdf
[4] J. Dongarra , Compressed Column Storage (CCS), Compressed
Column Storage (CCS), 20-Nov-1995. [Online]. Available at:
http://netlib.org/linalg/html templates/node92.html
[5] J. Fineman, M. Frigo, J. Gilbert, and C. Leiserson, Parallel Sparse Matrix-
Vector and MatrixTranspose-Vector Multiplication using Compressed
Sparse Blocks, Parallel Sparse Matrix-Vector and MatrixTranspose-
Vector Multiplication using Compressed Sparse Blocks. [Online]. Avail-
able at: http://gauss.cs.ucsb.edu/ aydin/talks/csb-spaa.pdf
[6] W. H. Press, Numerical recipes: the art of scientific
computing. Cambridge, UK: Cambridge University Press,
2007. P. Stathis, D. Cheresiz, S. Vassiliadis, and B.
Juurlink, Sparse Matrix Transpose Unit. [Online]. Available
at: https://www.aes.tu-berlin.de/fileadmin/fg196/publication/old-
juurlink/sparse matrix transpose unit.pdf
[7] W. Townsend, “Irregular Applications Sparse Matrix Vector Multiplica-
tion”, http://slideplayer.com/slide/9793914/, 2016
[8] V. der Vorst, Iterative Krylov Methods for Large Linear Systems.
Cambridge University Press