Scribd
Upload
23 views2 pages

IDS Lec4

Uploaded by

Arijit Chakrabarty
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
23 views2 pages

IDS Lec4

Uploaded by

Arijit Chakrabarty
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Geometric view of linear algebra

( we want to find a set of vectors, whose linear combination generates all the column vectors of a data
matrix)
Basis Vector: are a set of vectors that are linearly independent and spans(any vector of m-dimensional
space can be written as a linear combination) the m-dimensional space.

2D Space: R2 => (x,y)


Ex: v1 = [1 0]T v2 = [0 1]T
[2 1] = 2[1 0] + 1. [ 0 1] = 2v1 + 1v2

V1, v2 are basis vectors, linearly independent.

Basis vectors for a m-dimensional space is not unique.

The number of vectors in the basis set is same=> for spanning R2 , 2 basis vectors required
for spanning R3 , 3 basis vectors required

To span a subspace ( a subset of R2 ), less number of vector may suffice

Data Matrix : [ [c1] [c2] [c3] ….], where [ci] is a column vector.

Rank of matrix : r : => basis vector ( r linearly independent columns) ( in this case) => this vectors spans
the remaining columns.
Q: does the linearly independent vector span the entire space?

Gain in term of storage of the data matrix:


Data : 400 samples (columns), 4 attributes (rows) : 400 * 4 = 1600 elements to be stored
( in a rectangular matrix of size m x n, rank = rank( minimum of row , column))
Let row rank of this matrix is 2:
It is sufficient to store 2 * 4 = 8 values + 2 ( which vector I am choosing 1 or 2) * 400 = 808 values to
store 400 samples.

 Reduces the size of storage for the data matrix.

--
Distance of vectors, hyperplanes, halfspaces, eigenvalues and eigenvectors

Represent a line and plane:


R2
Line : ax1 + bx2 +c = 0
n1x1 + n2x2 + b = 0 n
nTX + b = 0, where n = [n1 n2]T; X = [x1 x2], n is normal vector. and X are vectors.

X2 X1(x11,x12)
Point X1 and X2 are on line.
nT X1 + b = 0;
nT X2 + b = 0;

X1
subtract second from one nT(X2 – X1) = 0 => orthogonal vectors ; (X2 – X1) gives direction of line. N is
perpendicular to a line.

This can be generalized for any m dimension.


3D space : R3

n = [ n1 n2 n3]T , n is perpendicular to a plane(2D)

Projection :
( reduce the m dimensional data to a smaller dimensional space for approximation)
Target : best approximation to represent data in lower dimension. From 3d space to 2 D space (
example)
Consider R2 space. Need 2 basis vectors v1 and v2
X( x1 x2 x3)
Projection: X’ of a vector X onto a lower dimension (2D) is define as:
X’ = c1V1 + c2V2 - eq 1
V2
X’
X = c1v1 + c2v2 + n ; n is a normal vector such that
nTv1 = v1Tn = 0
V1 nTv2 = v2Tn = 0 ; n vector is orthogonal to v1 as well as v2

v1Tn = 0
v1T(X - c1v1 + c2v2) = 0
v1TX = c1v1T v1
find value of c1
find value of c2 using v2Tn
X’ = substitute c1 and c2 in eq 1

Generalization : X’ = sum(j=1 to k) cjvj projection of X onto a space spanned by k linearly independent


basis vectors.

You might also like