Ai-900 2021

AI-900 Microsoft Azure AI
Fundamentals
Scott Duffy, Instructor
© 2021 Scott Duffy, softwarearchitect.ca… get the course for these slides at:
http://sjd.ca/ai900
January 2021 Updates
Microsoft Azure AI Fundamentals
“foundational knowledge of machine learning (ML) and artificial intelligence (AI)

concepts and related Microsoft Azure services”
● Candidates with non-technical backgrounds
● Candidates with a technical background who have a need to validate their

foundational level knowledge around AI and ML
● Describe AI workloads and considerations
● Describe fundamental principles of machine learning on Azure
● Describe features of computer vision workloads on Azure
● Describe features of Natural Language Processing (NLP) workloads on Azure
● Describe features of conversational AI workloads on Azure

You’ll be prepared
to take and pass
the AI-900 exam
But you don’t have
to, if you just want
to learn AI concepts
Easy to add AI
capabilities to your
own applications
What Azure AI Services Exist?
Vision Services
Speech Services
Language Services
Decision Services
Fundamentals
What Do We Mean By “AI”?
The term “Artificial
Intelligence” has
been used to
describe many
things
Oxford Dictionary
artificial intelligence (noun)
/ˌärdəˈfiSHəl inˈteləjəns/
“the theory and development of computer systems able to perform tasks that
normally require human intelligence, such as visual perception, speech recognition,
decision-making, and translation between languages.”
Incorporating
human intelligence
into machines
Artificial General Intelligence
That robot or computer in the movies that can teach itself a new task.
Common in science fiction. Does not currently exist in real life.
We’re not talking about that type of AI.

Narrow Artificial Intelligence
Computer systems which use human intelligence but have very strong limitations
in what they can do.
Siri, Cortana and Google Assistant are examples of Narrow AI.
“What is the weather today?”
“When is my next appointment?”
“Call Carol.”
Machine Learning
The study of computer algorithms that improve automatically through experience.
Unsupervised learning - ability to find patterns in data without human help
Supervised learning - humans label the data and gives general guidance
This course covers ML.

Natural Language Processing
Allows a machine to read and understand human language.
Machine translation, question answering, sentiment analysis, etc.
This course covers NLP.

Perception
The ability to use input from sensors - images, audio, lidar, sonar, radar, touch etc.
Covers things like facial recognition, speech recognition and object recognition.
Also covered in this course.

AI is “everywhere”.
AI-900 is a high-level exam
What is an ML Model?
Machine learning
(ML) allows
computers to use
data to forecast the
future...
… without
specifically being
programmed.
In ML, a model is a
program that can
be used to
recognize a pattern
in data
A model can be
used to predict
future behaviors
A model can be
used to categorize
something as one
thing or another
A model can be
used to recognize
people, objects and
landmarks using
unseen images
A model can be
used to understand
the context of
natural human text
or speech
You train a model
using “training
data”
You evaluate a
model using “test
data” to measure
how accurate is it
Once a model has
been deployed, it
can recognize
patterns in data it
has never seen
before
Fundamentals
Describe AI Workloads and
Considerations (15-20%)
Common AI Workloads
1. Prediction and Demand Forecasting
Prediction and Demand Forecasting
Prediction and Demand Forecasting
Using Machine Learning to Predict
● Give the machine all the relevant data you know
● Tell it for which field you want to predict
● It develops a model which it uses to make a prediction

Common Predictions
2. Anomaly Detection
3. Computer Vision
4. Natural Language Processing (NLP)
● What time are you open until today?
● When do you close?
● What time do you close today?
● What are your hours of operation today?
● Until which time can I come there to pick up my order?

Knowledge Mining - Cognitive Search
Ingest content from Blob storage, Tables, SQL Database, and Cosmos DB
PDFs, Images, Word Documents, Powerpoints, Audio
Uses AI tools such as image classification, face recognition, language detection,

key phrase extraction to create a searchable index
5. Conversational AI - Chat Bots
User: Do you have any of the Levi’s 501 Jeans in stock?
Chatbot: In what size?
User: Size 34 please
Chatbot: Yes, we have those. Do you want me to hold one aside for you?
User: Yes.
Chatbot: What is your name?
...
Fundamentals
Guiding Principles in AI
The prevalence of
AI causes some
ethical and moral
challenges
Unintended
consequences of
leaving important
decisions to a
computer
Unintended Consequences:
Decisions that are wrong
Decisions that are illegal (or at least, go against your own values)
Decisions that cannot be explained by anybody
Decisions that are harmful to society at large

Unintended Consequences
What If...
1. The traffic control dept installs sensors in every intersection to count cars
2. The road works dept chooses cameras instead of road sensors because it’s
cheaper to install and maintain
3. An IT intern uses AI to record the license plates, time and intersection of every
car the camera sees as an interesting experiment and leaves it running
4. The police request those records to find a robbery suspect
What If...
1. The traffic control dept installs sensors in every intersection to count cars
2. The road works dept chooses cameras instead of road sensors because it’s
cheaper to install and maintain
3. An IT intern uses AI to record the license plates, time and intersection of every
car the camera sees as an interesting experiment and leaves it running
4. The police request those records to find a robbery suspect
5. A hacker secretly has access to those records and posts
them to the Internet for anyone to use
Six Principles Should Guide AI Development
● Fairness
● Reliability and safety
● Privacy and security
● Inclusiveness
● Transparency
● Accountability
Principle of Fairness
“AI systems should treat everyone fairly

and avoid affecting similarly situated groups of people
in different ways.”
Imagine a hospital emergency room AI system

that prioritized patients in the order of their FICO credit score…
the more money you have, the faster you see a doctor.
Imagine a bank loan AI system

that gave more money to people named Michael
and less money to people named Alice.
Principle of Reliability and Safety
“To build trust, it's critical that AI systems

operate reliably, safely, and consistently
under normal circumstances
and in unexpected conditions.”
It’s so easy to program for the “sunny day”.
The data is never bad. No fields have blanks. No fields have accented characters.
All connected systems are functional.
There is no one trying to intentionally break the system.

How does the self-driving car operate at night?
In a rainstorm? In a snowstorm?
How does the military unmanned drone operate when the GPS is down?
How does the self-driving car operate when an object is approaching from the side
instead of the front or back?
Principle of Privacy and Security
Many countries and regions in the world are developing new standards and laws
to try to protect the data of its citizens. Laws are always slower than technology.
Six Principles Should Guide AI Development
● Fairness
● Reliability and safety
● Privacy and security
● Inclusiveness
● Transparency
● Accountability
Principle of Inclusiveness
“At Microsoft, we firmly believe

everyone should benefit from intelligent technology,
meaning it must incorporate and address a broad range
of human needs and experiences.”
Principle of Inclusiveness
How does a voice assistant work for someone who has a speech impediment?
How does a fitness tracker work for someone with a mobility disability?
How does “video-only” product instructions help someone who is blind?

Principle of Transparency
“When AI systems are used to help inform decisions

that have tremendous impacts on people's lives,
it is critical that people understand
how those decisions were made.”
Principle of Transparency
When someone is rejected by an AI system for a job, life insurance or a bank loan,
why were they rejected? If you are unable to tell them why, the system lacks
transparency and you can’t tell if it’s operating properly. It could be unfair, and you
won’t know it.
Principle of Accountability
“The people who design and deploy AI systems

must be accountable
for how their systems operate.”
Principle of Accountability
AI systems should not be the “final authority” in any decision that has a major
impact on people’s lives - employment, finances, health care, human safety, etc.
There should be regular review of how the AI is operating, and regular

improvement of the model.
Exam Tips
Really need to understand for which scenario
Fundamentals
Describe fundamental principles
of machine learning on Azure (30-
35%)
Common Machine Learning
Types
Regression
A type of supervised learning
The ability to predict the outcome variable

given 1 or more predictor variables.
Result is numeric - price, amount, size, etc.

Finds the
relationship
between the
variables
Classification
A type of supervised learning
Cluster analysis - assign a score to the odds

of it belonging to a cluster
What type of fruit is this?

Binary classification
only has two answers,
0 and 1
Multi-class
classifications
allow for other
options
Clustering
A type of unsupervised learning
Find groups of related things among data
What traits do my best customers

have in common?
Which is an Example of Regression?
A) Which movies to recommend to user Joe Smith (ie. Netflix)?
B) What make and model of car is in this image?
C) On a scale of 1-10, how happy is someone who makes $60,000 per year?
D) Is this email spam or ham?

Which is an Example of Regression?
A) Which movies to recommend to user Joe Smith (ie. Netflix)?
B) What make and model of car is in this image?
C) On a scale of 1-10, how happy is someone who makes $60,000 per year?
D) Is this email spam or ham?

Fundamentals
Core Machine Learning Concepts
Feature is an input
variable
Label is the thing
we’re predicting
Features and Labels in a Dataset
Given a pile of data, you (data scientist) need to determine which bits are relevant
to make decisions on
● Experiment
● Domain knowledge
● Keep in mind the principles of AI
Examples of Data
● First and last name
● Smoker or non-smoker (feature)
● Date of birth (feature)
● Marital status (feature)
● Gender (feature)
● Annual income (feature?)
● Blood type
● Number of children
● …
● Life expectancy (label)
Training the Model
The more input (historical) data you have, the more accurate the results
Don’t use ALL of your data to train the model
Divide your available labelled data into training and validation/test datasets
Ie. 1,000,000 rows of data - 500K to train the model, 500K to test the model
Split the training
and validation
datasets randomly
Evaluate the Results - Regression
Use the validation dataset to test the model, and measure how close or far the
actual results are from the predicted results
Mean Square Error
Large differences are much worse than small differences

Evaluate the Results - Classification
The result is to give a prediction score that the subject is part of the group
“70% confident this is an apple, 30% confident this is a pear”
So if an apple is mis-identified as a pear, that’s ok as long as it only happens 30%

of the time…
False Positives vs False Negatives
Compare true positives with false positives and true negatives with false
negatives when evaluating the model
How important is it to you that it never has a false positive?
Accuracy vs precision
Fundamentals
Core Machine Learning Tasks
Data ingestion and preparation
Steps
1. Load your data into Azure Storage from some other source
2. Clean up the data to remove low quality / bad data
3. Transform the data into useful / standard formats, intelligent defaults
4. Create a datastore in Azure ML to link to the data
5. Create a dataset in Azure ML to work with
a. File or table data
6. Register the dataset in your workspace

Data Ingestion
Azure Data Factory - the ELT solution
Azure ML Python SDK - scripts and custom code

Data Processing
Parquet and CSV file formats
Pandas dataframes for data less than 1GB
Ray, Dask, or Spark for data more than 1GB

Feature selection and engineering
Feature Selection is...
“The process of selecting a subset of relevant features … for use in model

construction.” - Wikipedia
Feature Selection
Azure ML supports both automated and manual configuration of ML
Feature selection is one of the core concepts and has a huge impact on the
performance of the model in ML
You need to identify and remove irrelevant or partially relevant features

Over-fitting the
training data
Simplify the model
Minimum redundancy,
maximum relevance
Curse of
dimensionality
Feature Engineering is...
“The process of using domain knowledge to extract features from raw data via
data mining techniques.” - Wikipedia
Deciding what
features to create
Improving existing
features by
transforming them
Brainstorming and
iterating on the best
features to achieve
the desired result
Model training and evaluation
Providing an ML
algorithm with
training data to
learn from
AutoML - try all the
algorithms to see
what is the best one
ML Designer - you
choose the
algorithm to use
Model Training is...
The process by which a machine learning algorithm “learns”
Depending on the type of ML (regression, classification, clustering)
Feed it the “training dataset”
Could take hours to process
May or may not be correct until you evaluate it.

Model evaluation is…
The process by which you estimate the accuracy of the model
You typically use a “test dataset” which is different than the “training dataset”
Compare the prediction made by the model with the actual result that you already
know
There are various ways to determine what is a good result and what is a bad result
Classification Model Metrics
Classification - Confusion Matrix
Confusion Matrix
True Positive (TP) - you predicted true, and it’s actually true (OK)
True Negative (TN) - you predicted false, and it’s actually false (OK)
False Positive (FP) - you predicted true, and it’s actually false (oops)
False Negative (FN) - you predicted false, and it’s actually true (oops)
Classification - Recall
Recall = TP / (TP + FN)
Out of all positive outcomes, how many

did you predict correctly?
Should be as high as possible

Classification - Precision
Precision = TP / (TP + FP)
Out of all positive predictions, how many

were actually positive

Classification - Accuracy
Accuracy = (TP + TN) / Total
How many correct predictions did you

get out of the total predictions?

Classification - F-Score or F-Measure
F-Score combines Recall and Precision into a single score.
F-score = 2 * recall * precision / (recall + precision)
Punishes extreme values

Regression Metrics
Calculating Regression Errors
R2 (R-squared) = measures the correlation between the observed outcome and
predicted outcome; the higher the better
RMSE (Root Mean Squared Error) = the average error between observed and
predicted; (the square-root of the mean square error); lower is better
RSE (Residual Standard Error) = accounts for the number of predictors in the
model; lower is better; not much different than RMSE
MAE (Mean Absolute Error) = similar to RMSE; less sensitive to

large errors than RMSE; lower is better
Model deployment and
management
Model deployment is...
The process of getting a trained model, that has passed evaluation, into a
production-ready state.
Thoroughly tested
the model
Choose the winning
“run”
Configure the
model for
production use
Publish and deploy
the model to a
compute
environment
Production URL
endpoint and
sometimes a key
Fundamentals
Capabilities of No-Code ML
Azure Machine Learning
“Azure Machine Learning is a cloud-based service

that helps simplify some of the tasks
and reduce the time it takes
to prepare data, train a model,
and deploy a predictive service.”
Automated
Machine Learning
(AutoML)
AutoML Steps
1. Identify the problem - classification, regression or time-series forecasting
2. Choose the environment - Python SDK or ML Studio
3. Specify the source and format of the labeled data
4. Configure the compute
5. Configure the AutoML parameters
6. Submit a training run
7. Review the results
Video Showing AutoML
Azure ML Designer
Another no-code
approach to model
design
Drag the dataset
onto the designer
canvas
Visualize the data
Exclude columns
that are not going
to be useful for
training
Clean rows with
missing data from
training
Normalize the data
to make it more
useful through
transformations
Drag training
models onto the
canvas
Training pipeline
can be converted to
the inference
pipeline
Fundamentals
Describe features of computer
vision workloads on Azure
(15-20%)
Computer Vision Features
Image Classification
Car
Object Detection
Object Detection
Semantic Segmentation
Optical Character Recognition
Facial Detection and Recognition
Fundamentals
Computer Vision Tools
In Azure, you can
use Computer
Vision service,
or you can use
Cognitive Services
Cognitive Services
includes many
other services
under one umbrella
All services require
a KEY and an
ENDPOINT
Computer Vision Service
Pre-trained ML model
Can recognize over 10,000 objects
Can generate automatic captions for images and tags
Content moderation for adult, racy or gory content
Detect faces
Text recognition
Custom Vision Service
A model that you can build and train
Classification or object detection
Upload an existing data set of images and classes
Publish the model so that you and others can use it

Custom Vision
service separates
out training and
prediction into two
resources
Face Service
Can recognize a human face in an image
Returns the rectangle coordinates of those 1 or more faces
Can recognize celebrities
Needs to be trained on your own data

Azure Face Service - Face Detection
Azure Face Service - Face Verification
To compare one face against a known face, to

identify if they belong to the same person.
Azure Face Service - Similar Faces
Azure Face Service - Face Grouping
Divides a set of unknown faces into groups

based on similarity. All faces of each group are
likely to belong to the same person.
Also supports differentiating by another factor

such as expression.
Azure Face Service - Identify API
Form Recognizer Service
Form Recognizer Service
Ideal for invoices and receipts
Pre-built model or custom model
Pre-built model recognizes common receipt formats, English in USA

Fundamentals
Describe features of NLP
workloads on Azure (15-20%)
NLP Workload Scenarios
Understands
written and spoken
language
Key Phrase Extraction
Identifying the main points of a document; context
“When I was shovelling snow in my driveway earlier today, I cut my finger. It wasn’t a
deep cut, and I will be ok. But it’s a bit painful. Snow isn’t fun sometimes.”
- Shovelling snow
- Cut my finger
- I will be ok
- A bit painful
- Snow isn’t fun
Entity Recognition
Identifying “entities” of a document; items categorized by type and subtype
“When I was shovelling snow in my driveway earlier today, I cut my finger. It wasn’t a
deep cut, and I will be ok. But it’s a bit painful. Snow isn’t fun sometimes.”
Snow, driveway, finger - objects
Earlier today - date
Driveway - location
Can also return
links to wikipedia
for relevant entries
I.e. “snow”
Sentiment Analysis
Pre-built ML model
Sentiment score from 0 to 1 - 1 being positive sentiment
“I thought the steak here was awesome and the staff was super friendly.”
Vs
“The steak was tough and the service was a bit slow. Not a great experience”.
Language Modeling
Build your own dictionary for terms in your industry
Train the language understanding engine for your own use

Speech Recognition and Synthesis
Speech recognition - the ability to detect and interpret speech
Speech synthesis - the ability to generate spoken output
Speech-to-text and text-to-speech

Translation
Supports over 60 languages
Getting better all the time
Understands semantic context
“Café da manhã” in Portuguese literally translates to

“coffee of the morning” in English,
but really means “breakfast” - the meal, nothing to do with coffee
NLP Workload Tools
Text Analytics Service
All of the Azure services that include key phrase extraction, entity detection, and
sentiment analysis
It can detect the language of text
Can detect when text has multiple languages and identify the predominant
language
Returns NaN when it can’t determine

Language Understanding Service (LUIS)
Understanding natural language
Three core concepts:
1. Utterances
2. Entities
3. Intents
Language Understanding - Utterances
An example of something a user will say:
“What time are you open until?”
“When do you close?”
“What time do you close today?”
“What is the latest you are open until?”

Language Understanding - Entities
An item to which an utterance refers:
● Today (DateTime)
● You (subject)
Language Understanding - Intents
The purpose or goal expressed by the user
Things your application is able to do
Goal: TodayHoursOfOperation
Language Understanding - Intents
Sometimes, there is no intent
Questions you don’t even want your AI to try to answer.
“What is the meaning of life?”
“How do teach a dog to walk himself?”

Speech Service
Text to speech
Speech to text
Audio translation
“Custom voice models”

Translator Text Service
Translate text between 70 languages
Custom models for specific industries / terms
Can detect the language being passed in

Fundamentals
Describe features of
conversational AI workloads on
Azure (15-20%)
Conversational AI
allows a human and
a computer to talk
Typically called
agents, or bots
Azure Bots can
operate over the
web, email, social
media, and voice
Do You Know You’re Talking to a Bot?
Conversational AI Use Cases
Webchat Bots
● Appear on a website or social media app
● Understands and responds to written text
● A limited scope of conversation
● Often friendly tone, called “chit chat”

Telephone Voice Menus
Speech recognition and speech synthesis
Conversational banking
Personal Digital Assistants
OK Google - when is my next appointment?
Hey Alexa - what time is the movie tonight?
Yo yo, Bing - tell Mom that I’ll be home this weekend from college
Conversational AI Services
QnA Maker Service
Automatically build a chat bot based on any semi-structured source
● Your website
● FAQ
● Product manuals
● etc
Azure Bot Service
Enterprise-grade chat bots
Start simply and grow in sophistication
Coded chat bots

Fundamentals
Thank you and best of luck!

Ai-900 2021

Uploaded by

Copyright:

Available Formats

Ai-900 2021

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ai-900 2021

Uploaded by

Copyright:

Available Formats

What are some examples of narrow AI?

What are some examples of narrow AI?

What is the difference between supervised and unsupervised machine learning?

What is the difference between supervised and unsupervised machine learning?

AI-900 Microsoft Azure AI

“foundational knowledge of machine learning (ML) and artiﬁcial intelligence (AI)

● Candidates with non-technical backgrounds

● Candidates with a technical background who have a need to validate their

● Describe fundamental principles of machine learning on Azure

● Describe features of computer vision workloads on Azure

● Describe features of Natural Language Processing (NLP) workloads on Azure

● Describe features of conversational AI workloads on Azure

Common in science ﬁction. Does not currently exist in real life.

We’re not talking about that type of AI.

Siri, Cortana and Google Assistant are examples of Narrow AI.

“What is the weather today?”

“When is my next appointment?”

Unsupervised learning - ability to ﬁnd patterns in data without human help

This course covers ML.

Machine translation, question answering, sentiment analysis, etc.

This course covers NLP.

Also covered in this course.

● Tell it for which ﬁeld you want to predict

● It develops a model which it uses to make a prediction

● When do you close?

● What time do you close today?

● What are your hours of operation today?

● Until which time can I come there to pick up my order?

PDFs, Images, Word Documents, Powerpoints, Audio

Uses AI tools such as image classiﬁcation, face recognition, language detection,

Chatbot: In what size?

User: Size 34 please

Chatbot: What is your name?

Decisions that cannot be explained by anybody

Decisions that are harmful to society at large

“AI systems should treat everyone fairly

Imagine a hospital emergency room AI system

Imagine a bank loan AI system

“To build trust, it's critical that AI systems

It’s so easy to program for the “sunny day”.

There is no one trying to intentionally break the system.

“At Microsoft, we ﬁrmly believe

How does “video-only” product instructions help someone who is blind?

“When AI systems are used to help inform decisions

“The people who design and deploy AI systems

There should be regular review of how the AI is operating, and regular

The ability to predict the outcome variable

Result is numeric - price, amount, size, etc.

Cluster analysis - assign a score to the odds

What type of fruit is this?

Find groups of related things among data

What traits do my best customers

B) What make and model of car is in this image?

D) Is this email spam or ham?

B) What make and model of car is in this image?

D) Is this email spam or ham?

Don’t use ALL of your data to train the model

Mean Square Error

Large differences are much worse than small differences