Tag: Principal Component Analysis Numerical Example Principal Component Analysis | Dimension Reduction Principal axis in science defi Share Watch on >» > '& Pattern Recognition Dimension Reduction- In patter recognition, Dimension Reduction is defined as- + Itis a process of converting a data set having vast dimensions into a data set with lesser dimensions. + Itensures that the converted data set conveys similar information concisely. Example- Consider the following example- + The following graph shows two dimensions x1 and x2. + x1 represents the measurement of several objects in cm. + x2 represents the measurement of several objects in inches.x2 (inches) x1 (cm) In machine learning, + Using both these dimensions convey similar information. + Also, they introduce a lot of noise in the system. ‘+ So, itis better to use just one dimension. Using dimension reduction techniques + We convert the dimensions of data from 2 dimensions (x1 and x2) to 1 dimension (21). + Itmakes the data relatively easier to explain. IH zDimension reduction offers several benefits such as- + It compresses the data and thus reduces the storage space requirements. + Itreduces the time required for computation since less dimensions require less computation + Itoliminates the redundant features. + Itimproves the model performance. Dimension Reduction Techniques- ‘The two popular and well-known dimension reduction techniques are~Dimension Reduction Techniques Principal Component Analysis Fisher Linear Discriminant Analysis (PCA) (LDA) 1. Principal Component Analysis (PCA) 2. Fisher Linear Discriminant Analysis (LDA) In this article, we will discuss about Principal Component Analysis. Principal Component Analysi + Principal Component Analysis is a well-known dimension reduction technique. + Ittransforms the variables into a new set of variables called as principal components. + These principal components are linear combination of original variables and are orthogonal. ‘+ The first principal component accounts for most of the possible variation of original data + The second principal component does its best to capture the variance in the data. + There can be only two principal components for a two-dimensional data set PCA Algorithm- ‘The steps involved in PCA Algorithm are as follows- Step-01: Get data, Step-02: Compute the mean vector (u). Step-03: Subtract mean from the given data.‘Step-05: Calculate the eigen vectors and eigen values of the covariance matrix. Step-06: Choosing components and forming a feature vector. Step-07: Deriving the new data set PRACTICE PROBLEMS BASED ON PRINCIPAL COMPONENT ANALYSIS- Problem-01:‘Compute the principal component using PCA Algorithm. OR Consider the two dimensional patterns (2, 1), (3, 8), (4, 3), (5, 6), (6, 7), (7, 8). ‘Compute the principal component using PCA Algorithm. OR ‘Compute the principal component of following data- CLASS 1 CLASS 2Teed Supporting people of all ages to Cone eat eee era Y=6,7,8 Solution- We use the above discussed PCA Algorithm- Get data. ‘The given feature vectors are- + x= (2,1) + x2=(3,5) + x= (4,3) xq = (5, 6) x5 = (6,7) x5 = (7,8)cea ere) Peet) cece rod Cea) healthier lives Step-02: Calculate the mean vector (). Mean vector (\)) =((2+34+4+54+6+7)/6,(14+5+3+6+7+8)/6) ACCESS = (4.5, 5)45 Mean vector (yu) = Step-03: ‘Subtract mean vector ()) from the given feature vectors. xy -U= (2-45, 1-5) = (2.5, -4) Xp -W= (3-45, 5-5) = (1.5, 0) (0.5, -2) X3-H=(4-4.5,3— + xg-u= (6-45, 6-5) = (0.5, 1) + Xs u= (6-45, 7-5) = (1.5, 2) + xg Y= (7-45, 8-5) = (2.5, 3) Feature vectors (x) after subtracting mean vector (1) are- Step-04: Calculate the covariance matrix. Covariance matrix is given by- alu VidicaeNow, m4 = (X4 - W(X - HE Mg = (Xz - H)(X2 - Ht mg = (xg - H)(xX3 - Ht mg = (xq - H)(X4 - Ht Mg = (Xs - H)(X5 - H)E mg = (xg - H)(Xg - H)* Now, Covariance matrix 05 a 25 0.5 0.5 oF 25 6.25 10 10 16 2.25 0 0 0 om 1 1 4 0.25 0.5 O57 pb 3 4 625 7.5 e318‘On adding the above matrices and dividing by 6, we get- 1 17.5 Covariance Matrix = — 6 Calculate the eigen values and eigen vectors of the covariance matrix. Covariance Matri: Step-05: ‘is an eigen value for a matrix M if itis a solution of the characteristic equation [M — All = 0. So, we have- 2.92 3.67 A OO - =0 3.67 5.67 oA 2.92 -A 3.67 =0 3.67 5.67-A From here, (2.92 -A\(5.67 —A) - (3.67 x 3.67) =0 16.56 — 2.92A—5.67A+ 2-13.47 =0Solving this quadratic equation, we get A= 8.22, 0.38, ‘Thus, two eigen values are Ay = 8.22 and Ap = 0.38, Clearly, the second eigen value is very small compared to the first eigen value So, the second eigen vector can be left out. Eigen vector corresponding to the greatest eigen value is the principal component for the given data set ‘So. we find the eigen vector corresponding to eigen value Ay We use the following equation to find the eigen vector- MX= AX where- + M= Covariance Matrix + X= Eigen vector = Elgen value ‘Substituting the values in the above equation, we get- Solving these, we get- 2.92X, + 3.67X, = 22K, B.87X + 5.87% = 22K ‘On simpification, we get- x1 8.22 x2From (1) and (2), Xq = 0.69X, From (2), the eigen vector is- x1 2.55 Eigen Vector : = x2 3.67 Thus, principal component for the given data set is- x1 2.55 Principal Component : = x2 3.67 Lastly, we project the data points onto the new subspace as- Projection Direction bv Ok HT DN &Problem-02: Use PCA Algorithm to transform the pattern (2, 1) onto the eigen vector in the previous question. Solution- The given feature vector is (2, 1). 2 Given Feature Vector : 1 The feature vector gets transformed to = Transpose of Eigen vector x (Feature Vector ~ Mean Vector) T = 2.55 2 45 x - 3.67 1 5 = [ 255 367 | x | 25 4 21.055 To gain better understanding about Principal Component Analysis, Watch this Video LectureSponsored Content Recommended by Genetic Variant Effects on Gene Expression Mapped in Elderly Human Brains GenomeWebCrenpat foe wee Beever ENVISIONING A FUTURE OF COMFORT WITH INNOVATIVE AND EFFICIENT TECHNOLOGY Enabling comfortable living Mitsubishi Electric