Scribd
Upload
3 views

Pattern_recognition

The document provides an overview of pattern classification, a field focused on categorizing data into distinct classes using various methods such as supervised and unsupervised learning. It discusses the importance of feature selection and reduction, as well as the steps involved in the classification process, including data collection, representation, and clustering. The article emphasizes statistical methods for pattern recognition while excluding fuzzy logic and rule-based methods.

Uploaded by

Mariana Diniz
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
3 views

Pattern_recognition

The document provides an overview of pattern classification, a field focused on categorizing data into distinct classes using various methods such as supervised and unsupervised learning. It discusses the importance of feature selection and reduction, as well as the steps involved in the classification process, including data collection, representation, and clustering. The article emphasizes statistical methods for pattern recognition while excluding fuzzy logic and rule-based methods.

Uploaded by

Mariana Diniz
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

An Introduction to Pattern Classification

Elad Yom-Tov
IBM Haifa Research Labs, University Campus, Haifa 31905, Israel
[email protected]

1 Introduction
Pattern classification is the field devoted to the study of methods designed to
categorize data into distinct classes. This categorization can be either distinct
labeling of the data (supervised learning), division of the data into classes (unsu-
pervised learning), selection of the most significant features of the data (feature
selection), or a combination of more than one of these tasks.
Pattern classification is one of a class of problems that humans (under most
circumstances) are able to accomplish extremely well, but are difficult for com-
puters to perform. This subject has been under extensive study for many years.
However during the past decade, with the introduction of several new classes of
pattern classification algorithms this field seems to achieve performance much
better than previously attained.
The goal of the following article is to give the reader a broad overview of
the field. As such, it attempts to introduce the reader to important aspects of
pattern classification, without delving deeply into any of the subject matters.
The exceptions to this rule are those points deemed especially important or
those that are of special interest. Finally, we note that the focus of this article
are statistical methods for pattern recognition. Thus, methods such as fuzzy
logic and rule-based methods are outside the scope of this article.

2 What Is Pattern Classification?


Pattern classification, also referred to as pattern recognition, attempts to build
algorithms capable of automatically constructing methods for distinguishing be-
tween different exemplars, based on their differentiating patterns.
Watanabe [53] described a pattern as ”the opposite of chaos; it is an entity,
vaguely defined, that could be given a name.” Examples of patterns are human
faces, handwritten letters, and the DNA sequences that may cause a certain
disease. More formally, the goal of a (supervised) pattern classification task is to
find a functional mapping between the input data X, used to describe an input
pattern, to a class label Y so that Y = f (X). Construction of the mapping is
based on training data supplied to the pattern classification algorithm. The
mapping f should give the smallest possible error in the mapping, i.e. the min-
imum number of examples where Y will be the wrong label, especially on test
data not seen by the algorithm during the learning phase.

O. Bousquet et al. (Eds.): Machine Learning 2003, LNAI 3176, pp. 1–20, 2004.

c Springer-Verlag Berlin Heidelberg 2004
2 E. Yom-Tov

An important division of pattern classification tasks are supervised as op-


posed to unsupervised classification. In supervised tasks the training data
consists of training patterns, as well as their required labeling. An example are
DNA sequences labeled to show which examples are known to harbor a genetic
trait and which ones do not. In unsupervised classification tasks the labels are
not provided, and the task of the algorithm is to find a ”good” partition of the
data into clusters. Examples for this kind of task are grouping of Web pages into
sets so that each set is concerned with a single subject matter.
A pattern is described by its features. These are the characteristics of the
examples for a given problem. For example, in a face recognition task some
features could be the color of the eyes or the distance between the eyes. Thus,
the input to a pattern recognition task can be viewed as a two-dimensional
matrix, whose axes are the examples and the features.
Pattern classification tasks are customarily divided into several distinct blocks.
These are:

1. Data collection and representation.


2. Feature selection and/or feature reduction.
3. Clustering.
4. Classification.

Data collection and representation are mostly problem-specific. Therefore it


is difficult to give general statements about this step of the process. In broad
terms, one should try to find invariant features, that describe the differences in
classes as best as possible.
Feature selection and feature reduction attempt to reduce the dimensionality
(i.e. the number of features) for the remaining steps of the task. Clustering
methods are used in order to reduce the number of training examples to the
task. Finally, the classification phase of the process finds the actual mapping
between patterns and labels (or targets). In many applications not all steps are
needed. Indeed, as computational power grows, the need to reduce the number
of patterns used as input to the classification task decreases, and may therefore
make the clustering stage superfluous for many applications.
In the following pages we describe feature selection and reduction, clustering,
and classification.

3 Feature Selection and Feature Reduction: Removing


Excess Data
When data is collected for later classification, it may seem reasonable to assume
that if more features describing the data are collected it will be easier to classify
these data correctly. In fact, as Trunk [50] demonstrated, more data may be
detrimental to classification, especially if the additional data is highly correlated
with previous data. Furthermore, noisy and irrelevant features are detrimental to
classification as they are known to cause the classifier to have poor generalization,

You might also like