Application of linear algebra in Computer Science and Engineering

What is linear algebra?

A branch of mathematics that is concerned with mathematical

structures closed under the operations of addition and scalar
multiplication and that includes the theory of systems of linear
equations, matrices, determinants, vector spaces, and linear
transformations is called linear algebra. Linear algebra is about linear
combinations. That is, using arithmetic on columns of numbers called
vectors and arrays of numbers called matrices, to create new columns and
arrays of numbers. Linear algebra is the study of lines and planes, vector
spaces and mappings that are required for linear transforms.

Linear algebra is used in most sciences and fields of engineering, because

it allows modeling many natural phenomena, and computing efficiently
with such models. For nonlinear systems, which cannot be modeled with
linear algebra, it is often used for dealing with first-order approximations,
using the fact that the differential of a multivariate function at a point is
the linear map that best approximates the function near that point.

History of linear algebra?

The study of algebra first emerged from the introduction of determinants.

Determinants were considered by Leibniz in 1693, and subsequently, in
1750, Gabriel Cramer used them for giving solutions of linear systems,
now called Cramer’s Rule. Later, Gauss further developed the theory of
linear systems by using Gaussian elimination. The study of matrix algebra
first emerged in England in the mid-1800s.

Why linear algebra is important?

Linear algebra is vital in multiple areas of science in general.

Because linear equations are so easy to solve, practically every area of
modern science contains models where equations are approximated
by linear equations (using Taylor expansion arguments) and solving for
the system helps the theory develop. Linear algebra provides concepts
 that are crucial to many areas of computer science, including graphics,
image processing, cryptography, machine learning, computer vision,
optimization, graph algorithms, quantum computation, computational
biology, information retrieval and web search.

Applic ation of linear algebra in Computer science & Engineering:

1. In graphics: Today Computer Graphics is being used in many fields.

Be it gaming industry, movies, architecture, in education sector for
making class room teaching interesting or for making presentations
for a business deal are some of them. It is interesting to know that
Linear Algebra a branch of mathematics has made all this possible.
In this paper we try to understand how images on the computer
screen are scaled, translated and rotated using simple matrix
transformations.
2. Statistics: The impact of linear algebra is important to consider,
given the foundational relationship both fields have with the field of
applied machine learning.

Some clear fingerprints of linear algebra on statistics and statistical

methods include:

 Use of vector and matrix notation, especially with

multivariate statistics.
 Solutions to least squares and weighted least squares, such
as for linear regression.
 Estimates of mean and variance of data matrices.
 The covariance matrix that plays a key role in multinomial
Gaussian distributions.
 Principal component analysis for data reduction that draws
many of these elements together.
Modern statistics and data analysis, at least as far as the interests of a
machine learning practitioner are concerned, depend on the
understanding and tools of linear algebra.
3. Machine Learning: Here we use linear algebra to Dataset and Data
Files, Images and Photographs, One-Hot Encoding, Linear
Regression, Regularization, Principal Component Analysis, Singular-
Value Decomposition, Latent Semantic Analysis, Recommender
Systems, Deep Learning.
4. Network: Linear algebra can be used to understand networks. A
network is a collection of nodes connected by edges and are also
called graphs. The adjacency matrix of a graph is defined by an array
of numbers. One defines the matrix entry Aij = 1 if there is an edge
from node i to node j in the graph. Otherwise the entry is zero. An
application is that one can read off the number of n-step walks in
the graph which start at the vertex i and end at the vertex j. It is
given by A n ij , where A n is the n-th power of the matrix A. You will
learn to compute with matrices as with numbers.
5. Data Comparison: Image, sound, and video compression standards
like PNG, MP3, and MP4 make use of the linear Fourier transform
and the fact that in the Fourier space, some parts of the information
can be cut away without disturbing the important parts of the
information. Typically a picture, a sound, or a movie is cut into
smaller chunks. These parts are represented by vectors. If U denotes
the Fourier transform and P is a cutoff function, then ~y = PU~x is
stored on some recording medium. The receiver reproduces the
recorded information by computing ˆx = T~y with an appropriate
transformation T. The vector ˆx is close to ~x in the sense that the
human eye or ear does not notice a big difference.
6. Cryptology: Cryptology much of current cryptological security is
based on the difficulty of factoring large integers n. One of the basic
factorization ideas going back to Fermat is to find integers x such
that x 2 (mod n) is a small square y 2 . Linear algebra (in particular,
Gaussian elimination) can be used effectively to find such integers.
 7. Games: Moving around in a world described in a computer game
requires rotations and translations to be implemented efficiently.
Hardware acceleration can help to handle this. Rotations are
represented by matrices having orthogonal properties. For example,
if an object located at (0,0,0) is rotated around the y-axis by an angle
ϕ, then every point in the object gets transformed by the matrix
[ cos(ϕ) 0 sin(ϕ), 0 1 0, − sin(ϕ) 0 cos(ϕ) ]
8. Markov Chains: Suppose we have three bags with 10 balls each.
Every time we throw a die and a 5 shows up, we move a ball from
bag 1 to bag 2. If the die shows 1 or 2, we move a ball from bag 2 to
bag 3. If 3 or 4 turns up, we move a ball from bag 3 to bag 1 and a
ball from bag 3 to bag 2. What distribution of balls will we see on
average? The problem defines a Markov Chain described by a matrix
[ 5/6 1/6 0, 0 2/3 1/3, 1/6 1/6 2/3 ] From this matrix, the equilibrium
distribution can be read off as an eigenvector of a matrix.
Eigenvectors will play an important role throughout the course.
9. Others: Camera modeling, epipolar geometry, calibration and self-
calibration, pose estimation, structure from motion. Audio, video
and image compression, including MP3, JPG & MPEF video.
Modulation and coding, including convolution codes and wi-fi,gigabit
Ethernet,HDTV & the GPS. Signal processing, including the fast
foutier transformation and autotune.

Conclusion: There are many applications of linear algebra in computer

science and engineering. From simple circuit solving to large web engine
algorithms. The heart beat of computer science is in linear algebra
especially in the generation and formation of computer coding schemes.
That is the study of matrices especially identity matrices under linear
algebra is key to computer coding schemes generation and formation
unlike what we have3 in computer engineering. These are programming
primitives normally encapsulated into programming languages
translations design.

Therefore linear algebra is a course in mathematics which is a subfield in

theoretical computer science. Of course linear algebra is also useful in
secondary programming like in computer graphics, database system and
IP address design and so on.