Scribd
Upload
16 views2 pages

New .........

Uploaded by

shalinikshalu6
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
16 views2 pages

New .........

Uploaded by

shalinikshalu6
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 2

Feature Selection

In this chapter the focus will be on an important component of dataset prep-


aration for data science: feature selection. An overused rubric in data science

circles is that 80% of the analysis effort is spent on data cleaning and prepa-

ration and only 20% is typically spent on modeling. In light of this it may

seem strange that this book has devoted more than a dozen chapters to

modeling techniques and only a couple to data preparation! However, data

cleansing and preparation are things that are better learned through experi-

ence and not so much from a book. That said, it is essential to be conversant

with the many techniques that are available for these important early
process

steps. In this chapter the focus will not be on data cleaning, as it was
partially

covered in Chapter 2, Data Science Process, but rather on reducing a dataset

to its essential characteristics or features. This process is known by various

terms: feature selection, dimension reduction, variable screening, key


parame-

ter identification, attribute weighting or regularization. Regularization was

briefly covered in Chapter 5 as applied to multiple linear regression. There it

was introduced as a process that helps to reduce overfitting, which is essen-

tially what feature selection techniques implicitly achieve. [Technically, there

is a subtle difference between dimension reduction and feature selection.

Dimension reduction methods—such as principal component analysis

(PCA), discussed in Section 14.2—combine or merge actual attributes in

order to reduce the number of attributes of a raw dataset. Feature selection

methods work more like filters that eliminate some attributes.]

First a brief introduction to feature selection along with the need for this pre-

processing step is given. There are fundamentally two types of feature selec-
tion processes: filter type and wrapper type. Filter approaches work by
selecting

only those attributes that rank among the top in meeting certain stated crite-

ria (Blum & Langley, 1997; Yu & Liu, 2003). Wrapper approaches work by

iteratively selecting, via a feedback loop, only those attributes that improve

the performance of an algorithm (Kohavi & John, 1997). Among the filter-

type methods, one can further classify based on the data types: numeric

versus nominal. The most common wrapper-type methods are the ones
associated with multiple regression: stepwise regression, forward selection,

and backward elimination. A few numeric filter-type methods will be

explored: PCA, which is strictly speaking a dimension reduction method;

information gainbased filtering; and one categorical filter-type method: chi-

square-based filtering.

You might also like