Phase 2 Ensemble Learning and Non-Generative Methods – Theory
Exercise 1 - Metrics
Cristhian Quiñonez
Grupo: 203008076A_1702
Tutor
Luis A. Anillo
UNIVERSIDAD NACIONAL ABIERTA Y A DISTANCIA – UNAD
MAYO DE 2024
Evaluation of a prediction model using classification metrics
Introduction
For a greater understanding of decision-making within the field of machine
learning, the data are used in an informative manner, allowing good use of them through
clearly combinable methods and techniques to find the best results in decision-making,
approximation. to circumstantial estimates according to certain established parameters that
allow us to find results with certainty through the analysis of information and with the help of
tools that allow us to approach reality in advance.
Through metrics we obtain an advantage by expanding knowledge and
evaluating the different types of existing methods to get closer to some type of expected
result.
Taking into account all these factors, we will use some metric tools in order to
establish in a standard way the obtaining of results through the domain of the data based on its
classification and organization, applying the foundations of the standard that will give us
assertively in numbers or variables the most suitable approach in remote decision making in
specific cases.
DEFINITIONS
Precision:
Emerges as a crucial metric in searching for model performance particularly
when the cost of a false positive is high
𝑇𝑃 + 𝑇𝑁
𝐴𝑐𝑐𝑢𝑟𝑎𝑟𝑦 =
𝑇𝑃 + 𝑇𝑁 + 𝐹𝑃 + 𝐹𝑁
For the previous expression we take into account that:
• TP are those that the algorithm classifies as positive
• TN are those that the algorithm classifies as negative (0)
• FP are those that the algorithm classifies as positive being actually negative
• FN are those that the algorithm classifies as negative being actually positive.
From the table, the classification metrics mentioned above are calculated: accuracy,
precision, recall and F1 score
ACURRACY
TP + TN
Accuracy =
TP + TN + FP + FN
184 +130
Accuracy =
23 + 184 +130 + 85
314
Accuracy =
422
Accuracy 0.744075
It is obtained that there is a great approximation with a significant percentage 74% of
representations of correct form similar to the predictions made, showing a good balance in the
performance of the prediction.
PRECISION
𝑇𝑃
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 =
𝑇𝑃 + 𝐹𝑃
130
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 =
130 + 83
130 65
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 = =
184 92
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 = 0,7065
Approximately 70,65%, which shows us that just as it is accurate and true, there is also a
small tendency for incorrect predictions to occur.
RECALL
𝑇𝑃
𝑅𝑒𝑐𝑎𝑙𝑙 =
𝑇𝑃 + 𝐹𝑁
130
𝑅𝑒𝑐𝑎𝑙𝑙 =
130 + 23
130
𝑅𝑒𝑐𝑎𝑙𝑙 =
153
𝑅𝑒𝑐𝑎𝑙𝑙 = 0,8496
F1 Score
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 𝑥 𝑅𝑒𝑐𝑎𝑙𝑙
𝐹1 𝑆𝑐𝑜𝑟𝑒 = 2.
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑅𝑒𝑐𝑎𝑙𝑙
0,7065 𝑥 0,8496
𝐹1 𝑆𝑐𝑜𝑟𝑒 = 2.
0,7065 + 0,8496
𝐹1 𝑆𝑐𝑜𝑟𝑒 = 0,7714
For F1 we find that it is approximately 0,7714. To get closer to the real value, there are
some points to take into account for continuous improvement.
CONCLUSION
We can say that it was found to have a high performance in precision and recovery,
intervening in the veracity of the model which is capable of accurately predicting the results
obtained in a positive and negative way.
The results obtained are taken into account for the analysis of future improvements in
accuracy that seek to establish better performance of the model and thus obtain maximum
performance for the benefit of a situation exposed to the standardization model.
REFERENCES
https://es.statisticseasily.com/exactitud-precisi%C3%B3n-recuperaci%C3%B3n-o-f1-
qu%C3%A9-m%C3%A9trica-prevalece/
https://www.themachinelearners.com/metricas-de-clasificacion/
