10 DAYS

NO CODE AI/ML
CHALLENGE

DAY 3
TASK 1
PROJECT CARD
AND DEMO

DAY 3
EASY ADVANCED
PROJECT CARD
GOAL:
• Build, train, test and deploy an Artificial intelligence
(AI) model to detect face masks.
3
TOOL:
• Google Teachable Machines

PRACTICAL REAL-WORLD APPLICATION:

• This project can be effectively used in indoor spaces
to detect if people are wearing facial masks.

DATA:
• INPUTS:
o Facial Images (Live data)
• OUTPUT:
o Mask Detection Status

Image Source: https://www.publicdomainpictures.net/en/view-image.php?image=320558&picture=coronavirus-covid-19-face-masks

PROJECT DEMO

3
TASK 2
BUSINESS CASE

DAY 3
EASY ADVANCED
BUSINESS CASE
• In the age of Covid-19 pandemic, wearing a mask in public is critical to limit
the spread of the virus and ensure public safety.
• Artificial Intelligence, Machine Learning and Deep Learning can automate the
3
process of detecting masks in public by looking at camera images and
extract proper features.
• These techniques can be used to ensure compliance with public health
guidelines and limit the spread of the virus.

• Link to article:
https://www.geekwire.com/2021/realnetworks-releases-first-hardware-product-ki
osk-checks-proper-facemask-usage/
READING TIME & QUIZ: AI APPLICATIONS
FOR FACIAL RECOGNITION
• Please read the article below and answer the following quiz.
o Link to Article:
3
https://www.thalesgroup.com/en/markets/digital-identity-and-securit
y/government/biometrics/facial-recognition

10 MINS

5 MINS
TASK 3
GTM DEMO: DATA
COLLECTION

DAY 3
EASY ADVANCED
GOOGLE TEACHABLE MACHINES: 1. DATA
COLLECTION
COLLECT LIVE DATA USING WEBCAM
(PLEASE MAKE SURE TO HAVE GOOD LIGHTING AND SOLID BACKGROUND).
3

Teachable Machine: https://teachablemachine.withgoogle.com/

TASK 4
GTM DEMO: MODEL
TRAINING

DAY 3
EASY ADVANCED
GOOGLE TEACHABLE
MACHINES: 2. MODEL TRAINING
TRAIN THE MODEL, CHANGE THE BATCH SIZE, LEARNING
RATE AND EPOCHS
3

Teachable Machine: https://teachablemachine.withgoogle.com/

TASK 5
GTM DEMO: DEPLOY &
EVALUATE MODEL

DAY 3
EASY ADVANCED
GOOGLE TEACHABLE LIVE DATA
MACHINES: 3. MODEL
EVALUATION & DEPLOYMENT
COLLECTION USING
WEBCAM FOR MODEL
TESTING. 3
CONFUSION
MATRIX AND
ACCURACY/LOSS
CURVES

Teachable Machine: https://teachablemachine.withgoogle.com/

GOOGLE TEACHABLE MACHINES: EXPORT AND UPLOAD

3
YOUR MODEL
3. MODEL EVALUATION &
DEPLOYMENT
GOOGLE TEACHABLE
MACHINES: 3. MODEL
EVALUATION & DEPLOYMENT 3
TEST THE
DEPLOYED MODEL
USING LIVE DATA
TRAINED NETWORK PERFORMANCE

3
TASK 6
CLASSIFIER MODELS
KPIs

DAY 3
EASY ADVANCED
CLASSIFICATION MODEL KPIs

o Classification Accuracy = (TP+TN) / (TP +

TN + FP + FN) TRUE CLASS
3
o Misclassification rate (Error Rate) = (FP + -
+
FN) / (TP + TN + FP + FN)

o Precision = TP/Total TRUE Predictions =

PREDICTIONS
TP/ (TP+FP) (When model predicted TRUE + TRUE + FALSE +
class, how often was it right?)

o Recall = TP/ Actual TRUE = TP/ (TP+FN)

(when the class was actually TRUE, how
often did the classifier get it right?)

FALSE - TRUE -
-
PRECISION Vs. RECALL EXAMPLE

TRUE CLASS FACTS:

100 PATIENTS TOTAL
3
91 PATIENTS ARE HEALTHY
+ - 9 PATIENTS HAVE CANCER
PREDICTIONS

• Accuracy is generally misleading and is not

+ TP = 1 FP = 1 enough to assess the performance of a
classifier.
• Recall is an important KPI in situations
where:
- FN = 8 TN = 90 o Dataset is highly unbalanced; cases
when you have small cancer patients
compared to healthy ones.

o Classification Accuracy = (TP+TN) / (TP + TN + FP + FN) = 91%

o Precision = TP/Total TRUE Predictions = TP/ (TP+FP) = ½=50%
o Recall = TP/ Actual TRUE = TP/ (TP+FN) = 1/9 = 11%
TASK 7
PRECISION VS.
RECALL

DAY 3
EASY ADVANCED
PRECISION DEEP DIVE
TRUE CLASS 3
+ -

PREDICTIONS
+ TP = 1 FP = 1

- FN = 8 TN = 90

NOTES:
o Precision is a measure of Correct Positives, in this example, the model predicted two
patients were positive classes (has cancer), only one of the two was correct.
o Precision is an important metric when False positives are important (how many times a
model says a pedestrian was detected and there was nothing there!
o Examples include drug testing
RECALL DEEP DIVE

TRUE CLASS
3
+ -

+ TP = 1 FP = 1
PREDICTIONS

- FN = 8 TN = 90
NOTES:
o Recall is also called True Positive rate or sensitivity
o In this example, I had 9 cancer patients but the model only detected 1 of them
o Important metric when we care about false negatives
o Example: Self driving cars and fraud detection
EXAMPLE 1: BANK FRAUD DETECTION

3
• Assume that you work as a data scientist at the bank and you have been assigned to develop a fraud
detection algorithm that could detect and block fraudulent transactions (binary classification problem).
o As a data scientist, which KPI would you optimize? Precision or recall?
o Plot a 2 x2 confusion matrix and list out various scenarios. TRUE CLASS

+ -
THERE WAS THERE WAS NO

PREDICTIONS
FRAUD AND FRAUD AND
“This is the only case MODEL MODEL
the bank loses money
so bank cares about
+ PREDICTED
FRAUD
PREDICTED
FRAUD
recall”

THERE WAS THERE WAS PISSED OFF

FRAUD AND NO FRAUD CUSTOMER
MODEL AND MODEL BUT THE
BANK LOSES
MONEY
- PREDICTED NO
FRAUD
PREDICTED NO
FRAUD
BANK IS OK!
EXAMPLE 2: SPAM EMAIL DETECTION

3
• Assume that you work as a data scientist at Google and you have been working on a project to detect spam
and ham e-mails using AI. The algorithm should be able to automatically detect/block spam emails.
o Which KPI would you optimize in this case? Precision or recall?
o Plot a 2 x2 confusion matrix and list out various scenarios. TRUE CLASS

+ -

PREDICTIONS
THERE WAS THERE WAS NO
SPAM EMAIL SPAM EMAIL AND
AND MODEL
+ PREDICTED
SPAM
MODEL PREDICTED
SPAM (BLOCKED
IT)
(BLOCKED IT)
“This is a case when
we care about
precision and it’s OK if THERE WAS A THERE WAS NO BLOCKED
we mess up recall a SPAM EMAIL AND SPAM EMAIL IMPORTANT
little bit” MODEL PREDICTED AND MODEL EMAILS (DREAM
- NO SPAM (WENT
TO INBOX)
PREDICTED NO
SPAM (WENT TO
JOB!)

NOT A BIG DEAL! INBOX)

TASK 8
FINAL PROJECT

DAY 3
EASY ADVANCED
FINAL PROJECT: MODEL IMPROVEMENT

1. In some scenarios, customers don’t properly wear facial masks. It’s critical
that trained AI/ML models are able to detect those cases.
3
2. Add 2 new classes to our project as shown below:
o Mask is ON but Nose not covered
o Mask is ON but Nose and mouth covered
3. Retrain the model by collecting at least 300 images per class
4. Tweak the model hyperparameters to enhance model performance
5. Analyze the confusion matrix, accuracy and loss graphs
6. Upload the model and deploy it
7. [Stretch goal] Improve the model generalization ability by including data
with different mask colors, lighting conditions...etc.
FINAL PROJECT: MODEL IMPROVEMENT

ADD 2 NEW CLASSES AND RETRAIN THE MODEL 3

FINAL PROJECT: MODEL IMPROVEMENT