Scribd
Upload
4 views8 pages

Wrapper and Filter 1

The document discusses feature selection methods, emphasizing the importance of identifying relevant features to improve model accuracy and reduce computational costs. It outlines univariate and multivariate methods, as well as filter and wrapper approaches for feature evaluation. Additionally, it describes search methods like sequential forward search and backward elimination for selecting optimal feature subsets.

Uploaded by

vincy shamleyeben
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
4 views8 pages

Wrapper and Filter 1

The document discusses feature selection methods, emphasizing the importance of identifying relevant features to improve model accuracy and reduce computational costs. It outlines univariate and multivariate methods, as well as filter and wrapper approaches for feature evaluation. Additionally, it describes search methods like sequential forward search and backward elimination for selecting optimal feature subsets.

Uploaded by

vincy shamleyeben
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 8

Feature Selection

(Filters, Wrapper Methods –


Block Diagram)
Reasons for Feature Selection
• Want to find which features are relevant
– Domain specialist not sure which factors are predictive of disease
– Common practice: throw in every feature you can think of, let feature
selection get rid of useless ones
• Want to maximize accuracy, by removing
irrelevant and noisy features
– For Spam, create a feature for each of ~105 English words
– Training with all features computationally expensive
– Irrelevant features hurt generalization
• Features have associated costs, want to
optimize accuracy with least expensive
features
– Embedded systems with limited resources
• Voice recognition on a cell phone
• Branch prediction in a CPU (4K code limit)
Terminology

• Univariate method: considers one variable


(feature) at a time
• Multivariate method: considers subsets of
variables (features) together
• Filter method: ranks features or feature
subsets independently of the predictor
(classifier)
• Wrapper method: uses a classifier to assess
features or feature subsets
Filtering
• Basic idea: assign score to each feature x
indicating how “related” x and the class y are

– Intuition: if x=y for all instances, then x is


great no matter what our model is; x contains
all information needed to predict y

• Pick the n highest scoring features to keep


Feature Selection Methods
Filter:
Selected
All Features Supervised Classifier
Features
Filter Learning
Algorithm Selected
Features

Wrapper:

Feature Feature Classifier


All Features Supervised Classifier
Subset Evaluation
Search Learning
Algorithm Criterion Selected
Features

Criterion Value
Feature Feature
All Features Supervised Classifier
Subset Evaluation
Search Learning
Algorithm Criterion Selected
Features

Criterion Value
Feature Feature
All Features Supervised Classifier
Subset Evaluation
Search Learning
Algorithm Criterion Selected
Features

Criterion Value

Search Method: sequential forward search

A B C D

A, B B, C B, D

A, B, C B, C, D
Feature Feature
All Features Supervised Classifier
Subset Evaluation
Search Learning
Algorithm Criterion Selected
Features

Criterion Value

Search Method: sequential backward elimination

ABC ABD ACD BCD

AB AD BD

A D

You might also like