Scribd
Upload
65 views13 pages

Confusion Matrix and Performance Evaluation Metrics

The document discusses performance metrics for evaluating classification models using a confusion matrix. The confusion matrix contains four quadrants representing true positives, false negatives, false positives, and true negatives. Key metrics discussed include accuracy, precision, recall, F1 score, true positive rate, false positive rate, and how these metrics can characterize classifiers as perfect, worst, ultra-liberal, or ultra-conservative.

Uploaded by

md shazzad
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
65 views13 pages

Confusion Matrix and Performance Evaluation Metrics

The document discusses performance metrics for evaluating classification models using a confusion matrix. The confusion matrix contains four quadrants representing true positives, false negatives, false positives, and true negatives. Key metrics discussed include accuracy, precision, recall, F1 score, true positive rate, false positive rate, and how these metrics can characterize classifiers as perfect, worst, ultra-liberal, or ultra-conservative.

Uploaded by

md shazzad
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 13

1

Confusion Matrix & Performance Measurement


Metrics
Al Amin Biswas
Lecturer, CSE, DIU
Confusion Matrix
2
 A confusion matrix for a two classes (+, -) is shown below.

 There are four quadrants in the confusion matrix, which are symbolized as
below.
 True Positive (TP: f++) : The number of instances that were positive (+) and
correctly classified as positive (+).
 False Negative (FN: f+-): The number of instances that were positive (+) and
incorrectly classified as negative (-).
 False Positive (FP: f-+): The number of instances that were negative (-) and
incorrectly classified as (+).
 True Negative (TN: f--): The number of instances that were negative (-)
and correctly classified as (-).
Confusion Matrix
3
Note:
 Np = TP (f++) + FN (f+-)
= is the total number of positive instances.

 Nn = FP(f-+) + Tn(f--)
= is the total number of negative instances.

 N = Np + Nn
= is the total number of instances.

 (TP + TN) denotes the number of correct classification

 (FP + FN) denotes the number of errors in classification.


4
Confusion Matrix Example

 For example,

Class + -
+ 52 (TP) 18 (FN)
- 21 (FP) 123 (TN)

Calculate the performance evaluation metrics


5 Accuracy
  It is defined as the fraction of the number of examples that are correctly
classified by the classifier to the total number of instances.
Accuracy
Performance Evaluation Metrics
 6
 

 We now define a number of metrics for the measurement of a classifier.

 In our discussion, we shall make the assumptions that there are only two
classes: + (positive) and – (negative)

 True Positive Rate (TPR): It is defined as the fraction of the positive


examples predicted correctly by the classifier.

 This metrics is also known as Recall, Sensitivity or Hit rate.

 False Positive Rate (FPR): It is defined as the fraction of negative examples


classified as positive class by the classifier.
Performance Evaluation Metrics
7
  False Negative Rate (FNR): It is defined as the fraction of positive
examples classified as a negative class by the classifier.

 True Negative Rate (TNR): It is defined as the fraction of negative


examples classified correctly by the classifier

 This metric is also known as Specificity.


8
Performance Evaluation Metrics
  
 Both, Precision and Recall are defined by
9
Performance Evaluation Metrics
 
F1 Score (F1): Recall (r) and Precision (p) are two
widely used metrics employed in analysis..
 It is defined in terms of (r or Recall) and (p or Precision) as
follows.

Note
F1 represents the harmonic mean between recall and precision

High value of F1 score ensures that both Precision and Recall


are reasonably high.
Analysis with Performance Measurement Metrics
10
 
 Based on the various performance metrics, we can characterize a classifier.

 We do it in terms of TPR, FPR, Precision and Recall and Accuracy

 Case 1: Perfect Classifier

When every instance is correctly classified, it is called the perfect classifier. In this case,
TP = P, TN = N and CM is
    Predicted Class

   
TPR = TP/(TP+FN)= =1 + -

+ P 0
FPR = =0
Actual
class

Precision = = 1 - 0 N

F1 Score = = 1
Accuracy = = 1
Analysis with Performance Measurement Metrics
11
  
 Case 2: Worst Classifier

When every instance is wrongly classified, it is called the worst classifier. In this
case, TP = 0, TN = 0 and the CM is
    Predicted Class

   
+ -
TPR = =0
+ 0 P

Actual
FPR = = 1

class
- N 0
Precision = = 0
F1 Score = Not applicable
as Recall + Precision = 0
Accuracy = =0
Analysis with Performance Measurement Metrics
12
  
 Case 3: Ultra-Liberal Classifier

The classifier always predicts the + class correctly. Here, the False Negative
(FN) and True Negative (TN) are zero. The CM is
    Predicted Class

   
+ -
TPR = = 1
+ P 0

Actual
FPR = = 1

class
- N 0
Precision =
F1 Score =
Accuracy = =0
Analysis with Performance Measurement Metrics
13
  
 Case 4: Ultra-Conservative Classifier

This classifier always predicts the - class correctly. Here, the False Negative
(FN) and True Negative (TN) are zero. The CM is
    Predicted Class

   
+ -
TPR = = 0
+ 0 p

Actual
FPR = = 0

class
- 0 N
Precision =
(as TP + FP = 0)
F1 Score =
Accuracy = =0

You might also like