Bayesian
Bayesian
Classification
Dr. Manish Kumar
Associate Professor
Chair: Data Analytics Lab & M.Tech (Data Engg.)
Department of Information Technology
Indian Institute of Information Technology-Allahabad, Prayagraj
Bayesian Classification
Bayesian Classification
What are Bayesian Classifiers?
▪ Statistical Classifiers
▪ Predict class membership probabilities
▪ Based on Bayes Theorem
▪ Naïve Bayesian Classifier
▪ Computationally Simple
▪ Comparable performance with DT and NN
classifiers
Bayesian Classification
▪ Probabilistic learning: Calculate explicit
probabilities for hypothesis, among the most
practical approaches to certain types of learning
problems
▪ Incremental: Each training example can
incrementally increase/decrease the probability that
a hypothesis is correct. Prior knowledge can be
combined with observed data.
Bayes Theorem
▪ Let X be a data sample whose class label is
unknown
▪ Let H be some hypothesis that X belongs to a
class C
▪ For classification determine P(H/X)
▪ P(H/X) is the probability that H holds given the
observed data sample X
▪ P(H/X) is posterior probability of H conditioned on
X
Bayes Theorem
Example: Sample space: All Fruits described by their
color and shape
X is “round” and “red”
H= hypothesis that X is an Apple
P(H/X) is our confidence that X is an apple given
that X is “round” and “red”
▪ P(H) is Prior Probability of H, ie, the probability
that any given data sample is an apple regardless
of how it looks
▪ P(H/X) is based on more information
▪ Note that P(H) is independent of X
Bayes Theorem
Example: Sample space: All Fruits
▪ P(X/H) ?
▪ It is the probability that X is round and
red given that we know that it is true
that X is an apple
▪ Here P(X) is prior probability =
P(data sample from our set of fruits is
red and round)
Estimating Probabilities
▪ P(X), P(H), and P(X/H) may be estimated
from given data
▪ Bayes Theorem
23