Scribd
Upload
41 views22 pages

Lecture 2 Part 1: Statistical Analysis (Bayesian Decision Theory, Probability Theory)

Statistical pattern recognition represents patterns as feature vectors and uses statistical techniques for classification. Bayesian decision theory provides a framework for designing classifiers to minimize expected risk by quantifying the tradeoffs between decisions using probabilities and costs. It involves prior probabilities, class conditional probabilities, posterior probabilities, and selecting the class that maximizes the posterior using the Bayes formula. The Bayes decision rule finds the decision boundary that partitions the feature space to minimize the average probability of error.

Uploaded by

Sakhawat Ali
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
41 views22 pages

Lecture 2 Part 1: Statistical Analysis (Bayesian Decision Theory, Probability Theory)

Statistical pattern recognition represents patterns as feature vectors and uses statistical techniques for classification. Bayesian decision theory provides a framework for designing classifiers to minimize expected risk by quantifying the tradeoffs between decisions using probabilities and costs. It involves prior probabilities, class conditional probabilities, posterior probabilities, and selecting the class that maximizes the posterior using the Bayes formula. The Bayes decision rule finds the decision boundary that partitions the feature space to minimize the average probability of error.

Uploaded by

Sakhawat Ali
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 22

Pattern Recognition (60014703-3)

Lecture 2 part 1
Statistical Analysis
(Bayesian Decision Theory, Probability Theory )

Instructor: Amany Al Luhaybi


1
Statistical Pattern Recognition…..
In statistical pattern recognition, a pattern is represented
by a set of features viewed as a dimensional feature
vector.
Well-known concepts from statistical decision theory
are utilized to establish decision boundaries between
pattern classes.
The data is reduced to vectors of numbers and statistical
techniques are used for the tasks to be performed.
The recognition system is operated in two modes:
training (learning) and classification (testing)
Decision theory
Decision theory is the study of making
decisions that have a significant impact

Decision-making is distinguished into:


Decision-making under certainty
Decision-making under non-certainty
Decision-making under risk
Decision-making under uncertainty
Bayesian Decision Theory
Design classifiers to make decisions subject to
minimizing an expected ”risk”.
The simplest risk is the classification error.
When misclassification errors are not equally
important, the risk can include the cost associated with
different misclassification errors.
Bayesian Decision Theory
Bayesian Decision Theory is a fundamental statistical
approach that quantifies the tradeoffs between various
decisions using probabilities and costs that accompany such
decisions.
Example: Patient has trouble breathing
– Decision: Asthma versus Lung cancer
– Decide lung cancer when person has asthma
Cost: moderately high (e.g., order unnecessary tests, scare patient)

– Decide asthma when person has lung cancer


Cost: very high (e.g., lose opportunity to treat cancer at early stage,
death)
Prior
Priori Probabilities or Prior reflects our
prior knowledge of how likely an event
occurs.
In the absence of any other information, a
random variable is assigned a degree of
belief called prior probability
Determine prior probabilities P(C1), P(C2)
e.g., 1209 samples:
221 #C =221 #C2=988
P(C ) 
1  0.183 1
1209
988
P(C2 )   0.817
1209
Class Conditional probability
When we have information concerning
previously unknown random variables then
we use conditional probabilities: P(a|b) the
probability of a given we know b.
Terminology
State of nature ω (class label):
e.g., ω1 for sea bass, ω2 for salmon

Probabilities P(ω1) and P(ω2) (priors):


e.g., prior knowledge of how likely is to get a sea bass or
a salmon

Probability density function p(x) (evidence):


e.g., how frequently we will measure a pattern with
feature value x (e.g., x corresponds to lightness)
Terminology (cont’d)
Conditional probability density p(x/ωj) (likelihood) :
e.g., how frequently we will measure a pattern with feature
value x given that the pattern belongs to class ωj

Conditional probability P(ωj /x) (posterior) :


e.g., the probability that the fish belongs to class ωj given
feature x.
Fish Sorting Example Revisited
Decision based on prior probabilities

• Favours the most likely class.


• This rule will be making the same decision all times.
Question
Consider a two-class problem, { c1 and c2 } where the
prior probabilities of the two classes are given by
P ( c1 ) = ⋅7 and P ( c2 ) = ⋅3
Design a classification rule for a pattern based only on
prior probabilities
Calculation of Error Probability – P ( error )
Solution
Decision based on class conditional
probabilities
Posterior Probabilities
Bayes Formula
Suppose the priors P(wj) and conditional densities p(x|
wj) are known,

prior
likelihood

p ( x |  j ) P( j )
P( j | x) 
p( x)
posterior evidence
Making a Decision
Probability of Error

Average probability of
error
P(error)

Bayes decision rule minimizes this error because


The dotted line at x0 is a threshold partitioning the
feature
space into two regions,R1 and R2. According to the
Bayes decision rule,for all values
of x in R1 the classifier decides 1 and for all values in
R2 it decides 2. However,
it is obvious from the figure that decision errors are
Example of the two regions R1 and R2 formed by the Bayesian
unavoidable.
classifier for the case of two equiprobable classes.

The dotted line at x0 is a threshold partitioning the feature space into two
regions,R1 and R2. According to the Bayes decision rule, for all values of x in
R1 the classifier decides 1 and for all values in R2 it decides 2. However, it is
obvious from the figure that decision errors are unavoidable.
Bayesian Decision Theory…

• Decision making relies on both the priors and the


likelihoods and Bayes Decision Rule combines them to
achieve the minimum probability of error.

You might also like