Evaluation AI Class 10 – 1 mark Questions

[1] Define: Evaluation

Ans.: Evaluation is the process of understanding the reliability of any AI model, based on
outputs by feeding the test dataset into the model and comparing it with actual answers.

[2] Name two parameters considered for the evaluation of a model.

Ans.: The two parameters considered for evaluation of a model are:

1. Prediction
2. Reality

[3] What is not recommended to evaluate the model?

Ans.: Its not recommended to use the data used to build the model to evaluate the model.

[4] Define overfitting.

Ans.: The model simply remember the whole training data set and will always predict the correct
label for any point in the training set. This is known as overfitting.

[5] Enlist the data sets used in AI modeling.

Ans.: There are two types of datasets used in AI.

1. Training Data Set
2. Testing Data Set

[6] What do you mean by prediction?

Ans.: Prediction refers to the output produced by the AI model.

[7] What is reality?

Ans.: Reality refers to the real scenario, when the prediction has been made by a model.

[8] What are the cases considered for evaluation?

Ans.:
1. True Positive
2. True Negative
3. False Positive
4. False Negative
[9] Write the term used for the following cases for heavy rain prediction:

Case Prediction Reality

1 Yes Yes

2 No No

3 Yes No

4 No Yes

Ans.:
1. True Positive
2. True Negative
3. False Positive
4. False Negative

[10] What do you mean by True Positive?

Ans.: True Positive refers to a condition occurs when both predictions done by AI model and
reality are True or Yes.

[11] What is True Negative?

Ans.: When both prediction and reality both are False or No, this condition is called True
Negative.

[12] What is a False Positive?

Ans.: When the prediction is predicted positive incorrectly and the reality is negative, this
condition is known as False Positive.

[13] What is False-negative?

Ans.: When the actual value predicted by AI model is false incorrectly and actual value is
positive, this condition is known as False negative.
[14] Ritika is learning evaluation. She wants to recognize the concept of
evaluation from the below-given facts:

1. A comparison between prediction and reality

2. Helps users to understand the prediction result
3. It is not an evaluation of matric
4. A record that helps in the evaluation

Help Ritika by giving the name to recognize the concept of evaluation.

Ans.: The concept is Confusion Matrix

[15] What is the need for a confusion matrix?

Ans.: The confusion matrix allows to understand the prediction results by an AI model.

[16] Devendra is confused about the condition when is the prediction said to be
correct, support your answer to help him to clear his confusion.

Ans.: If the prediction predicted by the AI model or machine matches with the reality, this is
known as the prediction said to be correct.

[17] Mentions two conditions when prediction matches reality.

Ans.: The two conditions when prediction matches reality are:

1. True Positive
2. True Negative

[18] Rozin is a student of class 10 AI. She wants to know the methods of
evaluation. Support her with your answer.

Ans.: The evaluation methods are:

1) Accuracy
2) Precision
3) Recall
4) F1 Score

[19] Mihir is trying to learn the formula of accuracy. What is the formula?

Ans: Accuracy = (True positives + True Negatives)/ (True positives + True negatives +
False positives + False negatives)

[20] If a model predicts there is no fire where in reality there is a 3% chance of

forest fire breaking out. What is the accuracy?

Ans.: The elements of the formula are as following:

1. True Positive: 0
2. True Negative:97
3. Total Cases: 100
Hence, Accuracy=(97+0)+100=97%

[21] What do you mean by precision?

Ans.: The percentage of true positive cases versus all the cases where the prediction is True is
known as prediction.

[22] Which cases are taken into account by precision?

Ans. Ture Positives and False Positives cases are taken into account by precision.

[23] Which cases are taken into account by the recall method?

Ans.: True Positives and False Negatives cases taken into account by recall method.

[24] Which measures are used to know the performance of the model?

Ans.: There are two measures used to know the performance of the model:
1. Recall
2. Precision

[25] Rohit is working on the AI model. He wanted to know the balance between
precision and recall. What it is?

Ans.: The balance between precision and recall is known F1 score.

[26] The task is to correctly identify the mobile phones as each, where photos of
oppo and Vivo phones are taken into consideration. Oppo phones are the positive
cases and Vivo phones are negative cases. The model is given 10 images of
Oppo and 15 images of Vivo phones. It correctly identifies 8 Oppo phones and 12
Vivo phones. Create a confusion matrix for the particular cases.

Ans.: The confusion matrix is as follows:

Prediction Prediction

Negative Positive

Reality Negative True Negative: 12 False Positive: 3

Reality Positive False Negative: 2 True Positive: 8

[27] There are some images of boys and girls. The girls are positive cases and
boys are negative cases. The model is given 20 images of girls and 30 images of
boys. The machine correctly identifies 12 girls and 23 boys. Create a confusion
matrix for the particular cases.

Ans.: The confusion matrix is as follows:

Prediction Prediction

Negative Positive

Reality Negative True Negative: 23 False Positive: 7

Reality Positive False Negative: 8 True Positive: 12

[28] There is data given for Facebook and Instagram users. The model is given
data for 200 Facebook users and 250 Instagram users. The machine identified 120
Facebook users correctly and 245 users of Instagram correctly. Create a
confusion matrix for the same.

Ans.: The confusion matrix is as follows:

Prediction Prediction

Negative Positive

Reality Negative True Negative: 120 False Positive: 80

Reality Positive False Negative: 5 True Positive: 245

[29] Consider that there are 10 images. Out of these 7 are apples and 3 are
bananas. Kirti has run the model on the images and it catches 5 apples correctly
and 2 bananas correctly. What is the accuracy of the model?

Ans.: Total correct predictions are: 5 + 2 = 7

Total predictions made: 5 + 2
So accuracy is: 7/7 = 100%.
The model does not predicted all of the images, but whatever predictions it makes are correct.
Hence accuracy is 100%.
[30] There are 16 images, 9 are cat images and 7 are dog images. The cat images
are positive cases and dog images are negative cases. The model identifies 5 cat
images correctly and 3 cat images as dog images. Similarly, it identifies 4 of them
correctly as dog images. Find the accuracy of the model.

Ans.: Total Predictions made: 5 + 3 + 4 = 12

Total Correct Predictions made: 5 + 4 = 9
So the accuracy is: 12/9 = 1.33 (Approx.)

[31] There are 20 images of aeroplanes and helicopters. The machine identifies 12
images correctly and 3 images incorrectly for aeroplanes. Similarly 2 images
correctly as helicopters. Find the accuracy of the model.

Ans.: No. of predictions made: 12 + 3 + 2 = 17

Total Correct predictions made: 12 + 2 = 14
Accuracy=17/14=1.21(Approx.)

[32] The prediction of the model is 1/4 and the recall of the same is 2/4. What is
the F1 score of the model?

Ans. F1 score= 2 x(precision x recall /precision + recall)

= 2 x (1/4×2/4)/(1/4+2/4)
= 1.3444/4

[33] Out of 300 images of Lions and Tigers, the model identified 267 images
correctly. What is the accuracy of the model?

Ans.: Accuracy = Total Correct Predictions / The no. of predictions = 267/300 = 0.89

[34] There are 400 images of fruits the AI model is able to predict correctly so the
accuracy of the model is exactly 0.5. How many correct predictions does the
machine make?

Ans.: Accuracy = Total Correct Predictions / The no. of predictions

0.5 = Total Correct Predictions / 400
Total Correct Prediction = 400 x 0.5 = 200
So the total correct predictions made by machine are 200.

[35] The recall comes 0.65 and the precision 0.70 for an AI model. What is the F1
score based on these metrics?

Ans.: F1 score = 2 x (Precision x Recall) / (Precision + Recall) = 2 x(0.65 x 0.70) / (0.65 +

0.70)=0.67
[36] The recall comes 0.80 and the precision 0.40 for an AI model. What is the F1
score based on these metrics?

Ans.: F1 Score= 2x(Precision x Recall) / (Precision + Recall) = 2 x( 0.80 x 0.40) / (0.80 + 0.40) =
0.53

Evaluation Class 10 Artificial Intelligence – 2

Marks Questions
[1] Explain the precision formula.

Ans.:The formula for precision is:

Precision = (True Positive / All Predicted Positives) x 100% = (TP / (TP + FP) ) x 100%

In this formula, the percentage of true positive cases values versus all the cases where
the prediction is true. It takes into account the True Positives and False Positives.

[2] Explain the recalled formula.

Ans.: The formula for recall is:

Recall = (True Positive / (True Positive + False Negative) ) x 100% = (TP / (TP + FN)) x 100%
In this recall formula, the fraction of positive cases which are identified correctly will be taken
into consideration. Here All True Positive cases and False Negative cases will be considered.

[3] What is the importance of evaluation?

Ans.: Evaluation is required to examine a model critically. It make the judgements about a
model to improve effectiveness and/or to inform programming decisions. It ensures that the
model is working properly and optionally. It helps to regulate what works well and what could be
better in a program. It is an initiative to appreciate how well it attains its goals.

[4] In which situation evaluation metric is more important for any case?

Ans.: F1 evaluation metric is more significant in any case. F1 score sort of upholds a balance
between is high again F1 score is high. The F1 score is a number between 0 and 1 and is the
harmonic mean of precision and recall. The formula to determine F1 score is:
F1 Score = 2 x ( Precision x Recall) / (Precision + Recall)

[5] Which value for the F1 score is the perfect F1 score? Explain with context.

Ans.: When both precision and recall had the value of 1 or 100% then the F1 score is also 1 or
100%. It is known as the perfect value for F1 score. The values for precision and recall falls
between 0 and 1 likely the F1 score values also falls between 0 and 1.
[6] Explain the evaluation metrics for mail spamming.

Ans.: In mail spamming, if the machine predicts that any email is spam email, then the person
will ignore that email. In this context sometime the person miss out the vital information. False
positive condition would have high cost as predicting the email as spam while the mail is not a
spam email.

[7] How evaluation metrics would be crucial for gold mining?

Ans.: A model predicts that there exists gem at a point and the individual keep on excavating
there but it turns out that it is a false apprehension. False positive condition is very costly as
predicting there is a treasure but in reality there is no treasure.

[8] How false-negative conditions will be hazardous in evaluation? Explain with

an example.

Ans.: The false negative conditions can be hazardous because of sometime the model did not
notice a condition which is very dangerous.

For example, a deadly virus has started scattering and the model which is supposed to
forecast a viral outbreak, does not notice it. The virus might spread extensively and
infect a lot of people.

[9] State and explain some possible reasons why AI model is not efficient.

Ans.: The possible reasons for AI model is not being effcient are:
1. Lacking of training data : This refers to the situation when data is not satisfactory as per
requirement or data is not used properly means something is missing from it, the model won’t
be efficient.
2. Unauthenticated Data/Wrong Data: For getting good results and predictions data must be
authenticated and correct. Unauthenticated data may not help in getting the good results.
3. Inefficient Coding/Wrong Algorithm: For preparing a good model, coding or algorithm is
necessary. The coding or algorithm should be written appropriately. If coding and algorithm is
not accurate or appropriate the model will not generate the desired output.
4. Less Accuracy

[10] High accuracy is not usable. Justify this with an example.

Ans.: High accuracy refers to accuracy 99.9%. This means that a small mistake leads to high
damage. High accuracy is very sensitive parameter for an AI model.

SCENARIO: An expensive robotic chicken crosses a very busy road a thousand times
per day. An ML model evaluates traffic patterns and predicts when this chicken can
safely cross the street with an accuracy of 99.99%.

Explanation: A 99.99% accuracy value on a very busy road strongly suggests that the
ML model is far better than chance. In some settings, however, the cost of making even
a small number of mistakes is still too high. 99.99% accuracy means that the expensive
chicken will need to be replaced, on average, every 10 days. (The chicken might also
cause extensive damage to cars that it hits.)

[11] High precision is not usable. Justify this with an example.

Ans.:
Example: Prediction of spam email whether it is spam or not.
In this case two conditions false positive and false negative may arise. The false positive refers
to the mail is predicted as “spam” but it is not a “spam”. False negative means the mail is
predicted as “Not Spam” but it is “Spam”. So many false negative condition will make the spam
filter ineffective but false positive may cause important mails to be missed.

[12] Suppose, the AI model has to bifurcate volleyball and football. Volleyballs are
positive cases and footballs are negative cases. There are 30 images of volleyball
and 26 images of footballs. The model has predicted 28 out of 30 volleyball and 2
volleyball as football. It predicts 24 footballs and 2 footballs as volleyball.
Compute accuracy and precision both.

Ans.:
Accuracy= correct predictions/total predictions made = (28 + 24) / (30 + 26 ) = 52/56 = 0.92
Precision = True Positives / (True Positives + False Positives) = 28 / (28 + 2) = 28 / 30 = 0.93

[13] There are 14 images of cows and buffalos. There are 8 images of cows and 6
images of buffalos. The model has predicted 5 cows and 4 buffalos. It identifies 1
cow as buffalo and 2 buffalos as cows. Compute the accuracy, precision, recall
and F1 score.

Ans.:
Accuracy = CP / TP = (5 + 4) / (8 + 6) = 9/14= 0.64
Precision = TP / (TP + FP) = 5 / (5 + 2) = 5/7 = 0.71
Recall = TP / (TP + FN) = 5 / (5+1) = 5/6 = 0.83
F1 Score = (2 x precision x recall) / (precision + recall) = (2 x 0.71x 0.83) / (0.71 + 0.83) =
1.1786/1.54 = 0.77
Or F1 Score = (2 x (5/7) x (5/6)) / ((5/7)+(5/6)) = (50/42) / (65/42) = 10/13

[14] For a model, the F1 score is 0.85 and precision is 0.82. Compute the recall for
the same case.

Ans.: In this questions F1 score is given and we have to find the recall value. So rearrange the
formula as following:
(1/F1 Score=(1/2) x ((1/Precision)+(1/Recall)) )So
(1/recall=((2/F1 Score)-(1/Precision)) or recall = (F1 Score x Precision) / (2 x precision – F1
Score)
recall= (0.85 x 0.82)/ (2 x 0.82 – 0.85) = 0.697/0.79 = 0.88
[15] Out of 100 pictures of buses and trucks, 80 are actually buses and 20 are
trucks. What is the minimum number of buses identified by a model to have a
recall of 90% or more?

Ans.: Here the recall value is 90% i.e. >=0.9.

Now, recall = Fractions of True Positive. There 80 buses identified correctly by model. Hence
the True Positive cases are 80. So required True Positive cases = 0.9 x 80 = 72. Hence 72
buses have to be correctly identified to have a recall value 90% or more.

[16] Out of 40 pictures of cats and rabbits, There are 25 rabbits, how many cats
most the model needs to identify correctly along with 15 rabbits images correctly
identified by the model, to achieve more than 75% accuracy?

Ans.: Lete there be x cats images correctly identified. Thus,

(15 + x) / 40 >= 0.75
or 15 + x >= 25
or x >= 10
Out of 40 images 25 are rabbits and 15 images already correctly identified as rabbit images.
Hence it must correctly identify 10 or more than cats images of the 15 images.

[17] Draw a confusion matrix for the following:

Positive/Negative: White Pages/ Yellow Pages No. of images: 150

Number of actual white pages images: 90 True Positives: 85

False Positives: 20 False Negatives: 25

Ans.: The confusion matrix is as follow:

True False

Positive (White Pages) 85 20

Negative (Yellow Pages) 5 25

[18] Find the F1 score from the given confusion matrix:

Ans.: The confusion matrix is as follow:

True False

Positive 44 6

Negative 8 15

Ans.:
Precision: TP / (TP + FP) = 44 / (44 + 6) = 44 / 50 = 0.88
Recall: TP / (TP + FN) = 44 / (44 + 15) = 44 / 59 = 0.746
F1 Score: 2 x precision x recall / (precision + recall) = 2 x 0.88 x 0.746 / (0.88 + 0.746) =
1.31296/1.626=0.81

Evaluation – Long Answer Questions (4 Marks)

[1] Shweta is learning NLP. She read about the F1 score but did not understand
the need for the F1 score formulation. Support her by giving an answer.

Ans.: The F1 score is also known as the F score or F measure of AI model’s test accuracy. It
calculated from the precision and recall of the test. Here
1) The precision is the number of correctly recognized positive results divided by the number of
all positive results, including the those not identified correctly.
2) The recall is the number of correctly identified positive results divided by the number of all
samples that should have been identified as positive.
3) The F1 score is defined as the harmonic mean of the test’s precision and recall. The formula
of F1 score is as below:
F1 score = 2 x precision x recall / (precision + recall)
From this formula,
i) A good F1 score means that you have high true positives and high true negatives. It means
that machine is correctly identifying real threats and not disturbs the users by false alarms.
ii) An F1 score is considered prefect when it’s 1 while the model is a total failure when it’s 0.
iii) F1 score is a better metric to evaluate the model on real-life classification problems and
when imbalanced class distribution exists.
[2] Calculate accuracy, precision, recall and F1 score for the following Confusion
Matrix. Suggest which metric would not be a good evaluation parameter and
why?

Reality:1 Reality:0

Prediction:1 50 30

Prediction:0 15 25

Ans.:
a) Accuracy: It is defined as the percentage of correct predictions out of all the observations.
Accuracy = (Correct Predictions / Total Cases) X 100% = ((TP + TN) / (TP + TN + FP + FN)) x
100%
= ((50 + 30) / (50 + 30 + 15 + 25) ) x 100% = (80 / 120) x 100% = 0.67

b) Precision: It is defined as the percentage of true positive cases versus all the cases where
the prediction is true.
Precision = (True Positive / All predicted positives) X 100% = TP / (TP + FP) x 100%
= 50 / (50 + 30) = 50/80 = 0.625

c) Recall: It is defined as the fraction of positive cases that are correctly identified.
Recall = True Positive / (True Positive + False Negative) = TP / (TP + FN)
=50 / (50 + 15) = 50 / 65 = 0.769

d) F1 Score: It is identified as the measure of balance between precision and recall.

F1 Score= (2 x precision x recall0 / (precision + recall) = 2 x 0.625 x 0.769 / (0.625 + 0.769)
= 0.96125 / 1.394 =0.7
Therefore,
Accuracy = 0.67
Precision = 0.625
Recall = 0.769
F1 Score = 0.7
By this results, recall is not good evaluation metric it needs to improve more. Two conditions are
very risky here:
1) False Positive: Suppose if we see the above confusion matrix for the heart attack cases then
a person is predicted with heart attack but does not have heart attack in reality.
2) False Negative: A person is predicted with no heart attack but in reality person is suffering
from the heart attack.
Hence False negative miss the actual patients. So recall needs more improvement.