Scribd
Upload
412 views3 pages

Machine Learning Evaluation

The classifier achieved an accuracy of 95.6% on the test data according to the confusion matrix. The precision for predicting yes was 98.3%. While most images of clear pavement were classified correctly, images that were zoomed in or out for pavement distresses were sometimes misclassified. The classifier performed poorly on these images because the training data did not include variations in zoom levels. Expanding the training data to include more variations in zoom levels could help address this issue.

Uploaded by

Khalida Muntasher
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
412 views3 pages

Machine Learning Evaluation

The classifier achieved an accuracy of 95.6% on the test data according to the confusion matrix. The precision for predicting yes was 98.3%. While most images of clear pavement were classified correctly, images that were zoomed in or out for pavement distresses were sometimes misclassified. The classifier performed poorly on these images because the training data did not include variations in zoom levels. Expanding the training data to include more variations in zoom levels could help address this issue.

Uploaded by

Khalida Muntasher
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

How well did your classifier work? Were you happy with the results?

What was the overall


percentage accuracy on test data?

In this study the classifier worked as desired; however, data cleaning is a bigger task
than inputting the data into a classifier and producing the output. I am happy with the results
obtained as the accuracy calculated using the confusion matrix was 95.6% and the precision
was 98.3%.

Accuracy: Overall, how often is the classifier correct?

(𝑇𝑃 + 𝑇𝑁) (235 + 4)


𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = = = 0.956
𝑇𝑜𝑡𝑎𝑙 250

where, TP – true positive and TP – true negative from the above equation it can be seen that
the accuracy obtained was 95.6% and hence the around 95.6% times the classifier worked
correctly.

Precision: When it predicts yes, how often is it correct?

𝑇𝑃 235
𝑃𝑟𝑒𝑐𝑖𝑠𝑜𝑛 = = = 0.983
𝑝𝑟𝑒𝑑_𝑦𝑒𝑠 239

From the above equation it is evident that the precision in predicting a yes when it is
correct is high (98.3%). Table 1 below provides more details on the same.

Table 1. Confusion Matrix


No of test samples - 250 Predicted Predicted Total

No Yes

Actual TN= 4 FP=4 8

No

Actual FN= 7 TP =235 242

Yes

Total 11 239 250


Describe what images your model classifies well and which they classify badly?

Almost all the images collected from the website were clear which the model was able
to classify. However, the images clicked by zooming in and out needed some attention. Only
the images for the pavement with distresses were clicked by zooming in to capture the
distresses. A slight reduction or increase in the area that was zoomed for certain images were
the ones that the classifier was not able to classify. Examples of misclassified images for
different distresses are represented below,

Fig 2. Misclassified images for different classes are shown on the


right side and respective images from training dataset are shown on
the left side
Explain why you think it performed well or badly on the images you described in the last
part?

The images of the normal pavement were clicked along the length of the pavement.
The image captured having single lane or two lanes, during the ongoing construction, the
movement of pedestrians walking, the movements of vehicles, footpaths, and road markings.
In this study, images of two different types of pavement was captured which includes earthen
and paved pavements. The classifier failed to detect the earthen pavement and the road
under construction having the shortest length. The trained model had all the roads of greater
length.

In the second part, images were collected related to the pavement cracks. The
pavement cracks were zoomed to capture the distresses. The classifier failed because the
pavement cracks captured near the road markings, or near movement of pedestrians and
movement of vehicles was not able to detect as this was misleading the trained model with
the normal pictures.

The third part, deals with the images having potholes (depressions on the pavement).
The potholes were zoomed and captured. Here, the trained model was not able to classify the
images because it was trained with the depth of depression of potholes. The images with flat
depressions and larger area were the ones that were not classified correctly.

Were there problems with the classifier that you were able to solve? Describe your strategy
for solving the problem

The dataset worked well with the classifier a such there were no problems. The classifier was
trained with 70% of the training data to attain the accuracy. It is required to have big data for
higher precision.

You might also like