Scribd
Upload
22 views

Probabilistic Classification Algorithms

Probabilistic classification algorithms, such as logistic regression and Bayes' theorem, provide a probability distribution over classes rather than a single class label. This accounts for prediction uncertainty. Bayes' theorem calculates the probability of an event based on prior probabilities and new evidence, and is commonly used for binary classification problems. Logistic regression estimates coefficients to model the relationship between independent variables and a binary outcome for classification.

Uploaded by

Nelson Librel Jr.
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
22 views

Probabilistic Classification Algorithms

Probabilistic classification algorithms, such as logistic regression and Bayes' theorem, provide a probability distribution over classes rather than a single class label. This accounts for prediction uncertainty. Bayes' theorem calculates the probability of an event based on prior probabilities and new evidence, and is commonly used for binary classification problems. Logistic regression estimates coefficients to model the relationship between independent variables and a binary outcome for classification.

Uploaded by

Nelson Librel Jr.
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 7

PROBABILISTIC

CLASSIFICATION
ALGORITHMS
Probabilistic classification algorithms are types of machine
learning model used for classification tasks that provide a probability
distribution over the different classes, rather than just a single class
label. This can be useful in situations where it is important to take
into account the uncertainty associated with the prediction.

Examples of probabilistic classification algorithms include logistic


regression and Bayes' theorem, also known as Bayes' rule.
◦ Bayes' theorem is a mathematical formula that describes
the probability of an event based on prior knowledge and
new information. It is named after Reverend Thomas
Bayes, who first formulated it in the 18th century. The
theorem is used to update the probability of a hypothesis in
light of new evidence.
Bayes' theorem can be written as:
◦ P(A|B) = P(B|A) * P(A) / P(B) is the general form of Bayes' theorem, where A
and B are events.

◦ P(y|X) = P(X|y) × P(y)|P(X) is a specific application of Bayes' theorem for


binary classification problems, where y is the class variable and X is the input
variable.
Bayes’ Theorem Example #1
◦ In this example, we are interested in finding the probability of a patient having liver disease (event
A) given that they are an alcoholic (event B). This is represented as P(A|B), which is the posterior
probability.
◦ We are given the following information:
• P(A) = 0.10, the prior probability of a patient having liver disease
• P(B) = 0.05, the prior probability of a patient being an alcoholic
• P(B|A) = 0.07, the probability of a patient being an alcoholic given that they have liver disease
◦ Using Bayes' theorem, we can calculate the posterior probability P(A|B) as follows:
◦ P(A|B) = P(B|A) * P(A) / P(B)
◦ Substituting the given values, we get:
◦ P(A|B) = (0.07 * 0.1) / 0.05 = 0.14
◦ So, the probability of a patient having liver disease given that they are an alcoholic is 0.14 (14%).
Logistic Regression Algorithm

◦ Logistic regression is used to predict a binary outcome, such as


success or failure, win or lose. It is a popular choice for
classification problems where the goal is to predict which of two or
more classes a given input belongs to. Logistic regression works by
estimating the coefficients of the logistic function that best fits the
data. This function models the relationship between the independent
variables and the binary outcome, and can be used to predict the
probability that a given input belongs to a particular class.
EXAMPLE
Let us use as an example our social network data set as presented in Table 9.4.
The age and the educational level are predictive attributes and the good/bad
company rating is the target attribute. A simple regression model that can be
obtained for this example is:
̂ l = −0.83338 + 0.03707 × Age − 0.13133 × Education Level.
Now let us apply this equation to a new instance. Andrew is 51 years old and
has an education level of 1.0. The logit is as follows:
̂ l = −0.83338 + 0.03707 × 51 − 0.13133 × 1.0 = 0.92535.
This value could be any value in IR. Using the logistic distribution function
we obtain:

You might also like