Feature Selection Method Type Basic Idea Techniques Pros
Correlation Analysis, Mutual
Rank features independently of the Fast & efficient, handle
Filter Methods Supervised Information, Univariate Statistical
model using statistical measures. high-dimensional data
Tests
Forward Selection, Backward
Search for best feature subsets using Often lead to high model
Wrapper Methods Supervised Elimination, Recursive Feature
a model, evaluating performance performance
Elimination (RFE)
Regularization (L1/Lasso,
Feature selection built into the model Efficient, consider feature
Embedded Methods Supervised L2/Ridge), Decision Trees/Random
training process interactions
Forests
Cons
May ignore feature
interactions, model-
agnostic
Computationally
expensive, risk of
overfitting
Dependent on the specific
model chosen