Scribd
Upload
94 views3 pages

Feature Extraction

Feature extraction algorithms can be linear or non-linear. Principal Component Analysis (PCA) is a common linear feature extraction method that uses an orthogonal transformation to convert data into a lower-dimensional space while maximizing variance. The algorithm for PCA involves preprocessing the data, calculating the covariance matrix to determine principal components, and transforming the data into the new feature space. Linear Discriminant Analysis (LDA) is a supervised feature extraction method that projects samples to maximize interclass differences and minimize intraclass differences. The algorithm for LDA involves calculating between-class and within-class scatter matrices to determine the optimal projection matrix.

Uploaded by

nandha shree
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
94 views3 pages

Feature Extraction

Feature extraction algorithms can be linear or non-linear. Principal Component Analysis (PCA) is a common linear feature extraction method that uses an orthogonal transformation to convert data into a lower-dimensional space while maximizing variance. The algorithm for PCA involves preprocessing the data, calculating the covariance matrix to determine principal components, and transforming the data into the new feature space. Linear Discriminant Analysis (LDA) is a supervised feature extraction method that projects samples to maximize interclass differences and minimize intraclass differences. The algorithm for LDA involves calculating between-class and within-class scatter matrices to determine the optimal projection matrix.

Uploaded by

nandha shree
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 3

Feature Extraction

Feature extraction algorithms might be linear or non-linear. Non-linear strategies


assume that the info of interest lie on an embedded non-linear manifold at intervals the
higher-dimensional space. If the manifold is of low enough dimension, we have a
tendency to visualize the info within the low-dimensional area. Thus, we have a
tendency to loosely classify non-linear strategies into two groups, those who offer
mapping and those that provides a visualization. Several of those non-linear spatial
property reduction ways are associated with the linear ways.

3.2.2.1 Principal Component Analysis

PCA is a common feature extraction method based on unsupervised dimension


reduction technique. In Figure 3.3 PCA uses an orthogonal transformation to convert
information into a lower-dimensional area whereas increasing the variance of the
information. The pre-processed dataset is taken as the input for feature extraction
mechanism for which PCA computes a covariance matrix which results in the selection
of optimum principle component. In the two-layer dimension reduction module PCA
transforms the n-dimensional random vector to the new features space consisting of
lower m dimensions. PCA will then sort the eigenvectors in descending order. At the
layer, Imbedded Error (IE) function is used to select the Principal Components (PC).
Cross validation is used to evaluate optimum principals with minimum errors.
Algorithm for PCA

Input: X- Pre-processed dataset

Output: Principal components

Step 1. Organize the data set

Step 2. Write x1…. xn as row vectors, each of which has p columns and place the
row vectors into a single matrix X of dimension n x p

Step 3. Calculate the empirical mean


1
3.1 m= ∑𝑛𝑘=1 𝑥𝑘
𝑛

Step 4. Find the covariance matrix


4.1 ∑𝑥 = ∑𝑛𝑘=1(𝑥𝑘 − 𝑚) (𝑥𝑘 − 𝑚)𝑇
Step 5. Eigenvector-eigenvalue decomposition
5.1 ∑ 𝑣 = λv
Step 6. Choose the principal components using IEF factor analysis measure

𝑙 ∑𝑚
𝑗=𝑙+1 𝜆 1⁄
6.1 IEF(l)= [ ] 2
𝑁𝑚(𝑚−𝑙)

Step 7. Compute the PCA transformed dataset


Linear Discriminant Analysis

LDA is another feature extraction method based on supervised dimension


reduction technique. The idea of LDA in Figure 3.4 given the training sample set, LDA
tries to project the sample onto a line so the projection points of interclass samples are
as close as attainable and therefore the projection points of the intraclass samples are as
far apart as attainable. The output obtained from PCA is further optimized using LDA
to apply the labelled data obtained from samples of PCA transformed the dataset to
achieve a reasonable speed in the proposed intrusion reduction system. There are two
scatter matrices that need to be obtained in LDA, the Between class Scatter matrix (SB)
and the Within class Scatter matrix (SW). The ratio between the projection matrix of
SB and SW is obtained for an ideal transformation of the processed dataset.

Algorithm for LDA

Input: PCA transformed dataset

Output: LDA transformed dataset

Step 1. The first step is to calculate the between-class variance


1.1 𝑆𝐵 = ∑𝑐(𝑢𝑐 − x)(𝑢𝑐 − 𝑥)𝑇

Step 2. Calculate the instance between the mean and the sample of each class
2.1 SW = ∑c ∑iεc(xi -uc )(xi − uc )T
Step 3. Find the mean value of class C
1
3.1 uc = ∑xεci x
ni

Step 4. Compute the ratio of SB an SW


WT
r SB Wr
4.1 J =
WT
r SW Wr

Step 5. Project the LDA transformed dataset.

You might also like