DEPARTMENT OF ELECTRONICS &

COMMUNICATION ENGINEERING

FEATURE SELECTION TECHNIQUES

ALOK KUMAR
OUTLINE:
 INTRODUCTION
 NEED OF F.S.
 F.S. TECHNIQUES
 F.S. FOR BIOMEDICAL IMAGING
 CONCLUSION
 REFERENCES
WHAT ARE FEATURES ?
 The information extracted from the data is known as
features.
 Features may be specific structures in the image such
as points, edges or objects.
CLASSIFICATION OF FEATURE
 Type: 1 Relevance: contain useful information
 Type: 2 Irrelevance: provide no useful information
 Type: 3 Redundance: provide no extra information,
linearly correlated with Type-1 features.
 e.g. X = 1 meter, Y = 100 cm, Z = 3.28 feet,
M = 39.37 inches
 NEED OF FEATURE SELECTION
 A grayscale image of 100 x 100 resolution can have 50
million features.
 More features = more information about the object
= better classification power
Inclusion of Irrelevant and
redundant features :
1. increase the demand of
storage memory
2. require large computation
time.
Contd..
 Feature Selection or dimensionality reduction is an
important pre-processing step to select the relevant
features.
 Advantages of Feature Selection:
 Improving the performance of the machine learning
algorithms.
 Data understanding, gaining knowledge about the process
and perhaps helping to visualize it.
 General data reduction, limiting storage requirements and
perhaps helping in reducing costs.
 Feature set reduction, saving resources in the next round of
data collection or during utilization.
 Simplicity, possibility of using simpler models and gaining
speed.
FEATURE SELECTION vs.
FEATURE EXTRACTION
FEATURE SELECTION APPROACHES
There are two approaches in feature selection
 Forward Selection: Start with no variables and add
them one by one.
 Backward Selection: Start with all the variables and
remove them one by one.
 FEATURE SELECTION METHODS
 1. FILTER METHOD: Features are selected on the basis of
their scores/rank in various statistical tests for their
correlation with the outcome variable.
 2. WRAPPER METHOD: Feature selection process is
based on a specific machine learning algorithm. It follows
a greedy search approach.
 3. EMBEDDED METHOD: feature selection is an integral
part of the classification model. Combines the advantages
of filter and wrapper method.
COMPARISON
BIOMEDICAL IMAGING
 Biomedical Imaging plays a vital role in the global
healthcare system as it is used for fast and effective
diagnosis of several diseases like brain tumor, lung
cancer, breast cancer etc.
 Different image modalities such as Computed
Tomography (CT), Digital Mammogram (DM),
Ultrasound (US), Magnetic Resonance Imaging (MRI)
etc. are used for screening the different parts of human
body.
 Lung cancer turned into the prevalent reason for
disease occurrence and mortality, with 2.1 million
(11.6%) new lung malignancy cases and 1.8 million
deaths predicted in 2018.
LUNG CANCER
There are two principle kinds of lung
cancer: small cell lung cancer (SCLC)
and non-small cell lung cancer
(NSCLC).
Small lesions cannot be identified by X-
rays and are just perceptible by a CT
scan.
Detection of lung cancer at early stage
can help the doctors to start the
treatment on time, which can save the
life of a patient.
 FEATURE SELECTION FOR
BIOMEDICAL IMAGING
 Cancer nodule usually has large number of features. It
is important to identify and extract interesting
features form it.
 Aim of feature selection is to find a set of features that
define the shape of nodule as precisely and uniquely.
 As feature selection methods can improve the
classification accuracy, it is helpful to differentiate
between benign and malignant nodule.
 Various Methods of F.S. for lung cancer detection
comparison.docx
CONCLUSION
 As we are in the digital era every moment it generates
million, billion of data. This increases the burden for
processing, which in turn affect the decision making on any
application.
 Filter-based feature selections are computationally faster
when compared with the wrapper method and less
accurate. Whereas in wrapper method the accuracy is more
but computationally costlier.
 Dimension reduction provides several advantages: it results
in a low dimensional model, requires less memory space,
reduces the risk of overfitting, better accuracy and reduces
the time complexity.
REFERENCES
 1. Yu, L., H. Liu. Efficient Feature Selection via Analysis of Relevance and Redundancy. – J.
Mach. Learn. Res., Vol. 5, 2004, No Oct, pp. 1205-1224.
 2. Gheyas, I. A., L. S. Smith. Feature Subset Selection in Large Dimensionality Domains. –
Pattern Recognit., Vol. 43, January 2010, No 1, pp. 5-13.
 3. Yang, Y., J. O. Pedersen. A Comparative Study on Feature Selection in Text Categorization.
– In: Proc. of 14th International Conference on Machine Learning, ICML’97, 1997, pp. 412-
420.
 4. Yan, K., D. Zhang. Feature Selection and Analysis on Correlated Gas Sensor Data with
Recursive Feature Elimination. – Sensors Actuators, B Chem., Vol. 212, Jun 2015, pp. 353-
363.
 5. Jain, A., D. Zongker. Feature Selection: Evaluation, Application, and Small Sample
Performance. – IEEE Trans. Pattern Anal. Mach. Intell., Vol. 19, 1997, No 2, pp. 153-158.
 6. Gutkin, M., R. Shamir, G. Dror. SlimPLS: A Method for Feature Selection in Gene
Expression-Based Disease Classification. – PLoS One, Vol. 4, July 2009, No 7, p. e6416.
 7. Ang, J. C., A. Mirzal, H. Haron, H. N. A. Hamed. Supervised, Unsupervised, and Semi-
Supervised Feature Selection: A Review on Gene Selection. – IEEE/ACM Trans. Comput.
Biol. Bioinforma., Vol. 13, September 2016, No 5, pp. 971-989.
 8. Bins, J., B. A. Draper. Feature Selection from Huge Feature Sets. – In: Proc. of IEEE Int.
Conf. Comput. Vis., Vol. 2, 2001, pp. 159-165.
 9. Ferri, F., P. Pudil. Comparative Study of Techniques for Large-Scale Feature Selection. –
Pattern Recognit. Pract. IV, Vol. 1994, 1994, pp. 403-413.
 10. Pudil, P., J. Novovičová, J. Kittler. Floating Search Methods in Feature Selection. – Pattern
Recognit. Lett., Vol. 15, November 1994, No 11, pp. 1119-1125.
 11. Doak, J. An Evaluation of Feature Selection Methodsand Their Application to Computer Security. CSE-
92-18, 1992. 82 p.
 12. Skalak, D. B. Prototype and Feature Selection by Sampling and Random Mutation Hill Climbing
Algorithms. – In: Proc. of 11th International Conference on Machine Learning, 1994, pp. 293-301.
 13. Goldberg, D. E. Genetic Algorithms in Search, Optimization, and Machine Learning. Boston, MA, 1989.
– Read. Addison-Wesley, 1989.
 14. Brassard, P., Gilles, Bratley. Fundamentals of Algorithmics. Englewood Cliffs, NJ, Prentice Hall, 1996.
 15. Glover, F. Future Paths for Integer Programming and Links to Artificial Intelligence. – Comput. Oper.
Res., Vol. 13, January 1986, No 5, pp. 533-549.
 16. Li, B., L. Wang, W. Song. Ant Colony Optimization for the Traveling Salesman Problem Based on Ants
with Memory. – In: Proc. of 4th International Conference on Natural Computation, 2008, pp. 496-501.

 17. Nozawa, H. A Neural Network Model as a Globally Coupled Map and Applications Based on Chaos.
Chaos an Interdiscip. – J. Nonlinear Sci., Vol. 2, July 1992, No 3, pp. 377-386.
 18. Luonan, C., K. Aihara. Chaotic Simulated Annealing by a Neural Network Model with Transient Chaos.
– Neural Networks, Vol. 8, 1995, No 6, pp. 915-930.
 19. Wang, L., S. Li, F. Tian, X. Fu. A Noisy Chaotic Neural Network for Solving Combinatorial
Optimization Problems: Stochastic Chaotic Simulated Annealing. – IEEE Trans. Syst. Man, Cybern. Part B
Cybern., Vol. 34, 2004, No 5, pp. 2119-2125.
 20. Narendra, P. M., K. Fukunaga. A Branch and Bound Algorithm for Feature Subset Selection. – IEEE
Trans. Comput., Vol. C-26, 1977, No 9, pp. 917-922.
 21. Land, A., A. Doig. An Automatic Method of Solving Discrete Programming Problems. – Econometrika,
Vol. 28, 1960, No 3, pp. 497-520.
 22. Poli, R., J. Kennedy, T. Blackwell. Particle Swarm Optimization. – Swarm Intell., Vol. 1, October 2007,
No 1, pp. 33-57.
 23. Dash, M., H. Liu. Feature Selection for Classification. – Intell. Data Anal., Vol. 1, January 1997, No 1-
4, pp. 131-156.
 24. Fayyad, M. U., K. B. Irani.The Attribute Selection Problem in Decision Tree Generation. – Aaai-92,
1992, pp. 104-110.
 25. Liu, H., R. Setiono. A Probabilistic Approach to Feature Selection – A Filter Solution. – In: Proc. of 13th
International Conference on Machine Learning, 1996, pp. 319-327.
THANK YOU

Any Questions?