Azure OpenAI Service

• Quick Overview
• How Azure OpenAI Service process data
• Security, Governance, and Controls
• Use Cases
• Quick Demo
• Q&A
1956
Artificial Intelligence

Machine Learning
1997

Deep Learning
2017

Generative AI
2021
 Ensure that artificial Empower every person and
general intelligence (AGI) organization on the planet
benefits humanity. to achieve more

GPT-3 Codex DALL·E

Generate and Understand Text Generate and Understand Code Generate images from text prompts
 Azure AI

Applications Partner Solutions

Business
Users
Application Platform
AI Builder
Power BI Power Apps Power Automate Power Virtual Agents

Scenario-Based Services
Applied AI Services
Bot Service Cognitive Search Form Recognizer Video Indexer Metrics Advisor Immersive Reader

Customizable AI Models
Cognitive Services Developers &
Vision Speech Language Decision OpenAI Service Data Scientists

ML Platform Azure Machine Learning

Azure OpenAI Service

GPT-3 Codex

DALL·E (preview)
Prompts

Show and tell provide quality data check your settings

Tokens

Embeddings
vector of floating-point
numbers

cosine similarity

Completions
simple but powerful text-in, text-out
interface to any of supported models

Responsible AI - Content Filtering

Fine-tuning
 Generative AI

GPT-3 Codex DALL·E

Prompt: Prompt: Prompt: A white Siamese cat

Write a tagline for an ice Table customers, columns = Response:

cream shop. [CustomerId, FirstName,
LastName, Company, Address,
Response: City, State, Country,
PostalCode]
We serve up smiles with every
scoop! Create a SQL query for all
customers in Texas named Jane
query =

Response:

SELECT *
FROM customers
WHERE State = 'TX' AND
FirstName = 'Jane'
 Azure OpenAI | GPT-3 Family of Models
Model Request Description, performance, cost Use cases
Davinci 4,000 tokens Most capable GPT-3 model. Can do Complex intent, cause and
any task the other models can do, effect, summarization for
often with higher quality, longer output audience
and better instruction-following.

Curie 2048 tokens Very capable, but faster and lower cost Language translation,
than Davinci. complex classification, text
sentiment, summarization

Babbage 2048 tokens Capable of straightforward tasks, very Moderate classification,

fast, and lower cost. semantic search classification

Ada 2048 tokens Capable of very simple tasks, usually Parsing text, simple
the fastest model in the GPT-3 series, classification, address
and lowest cost. correction, keywords
Azure OpenAI | Sample Use Cases

Customer conversation

Azure OpenAI Instruction

Output
Azure OpenAI | Sample Use Cases

Input - German

Azure OpenAI
Instruction - English

Output - English
Azure OpenAI | GPT-3 Prompt Design

Prompt – Text input that Completion – Output that

provides some context to the GPT-3 generates based on
engine on what is expecting. the prompt.
 2 min demo

Azure OpenAI | Demo

Enabling all skill levels to build prototypes easily and explore the art of possible
 Language
Azure OpenAI Reason over structured and unstructured data:
Classification, Sentiment, Entity Extraction, Search
Summarization
• Call center call transcripts
Service Use Cases • Refined text-based search with embeddings • Subject Matter Expert Documents
• Product feedback sentiment • Competitive analysis
• Customer and employee feedback classification • Peer Analysis
• Claims and risk analyses • Technical reports
• Support emails and call transcripts • Product and service feedback
• Social media trends • Social media trends
Codex Writing Assistance
• Natural Language to Code • Marketing copy / email taglines Conversational AI

• Natural Language to SQL • Long format text • Smart assists for call centers

• Code to Natural Language • Paragraphs from bullets • Tech support chat bots

• Code documentation • Virtual assistants

Use Cases that use multiple model capabilities • Artificial humans
• Refactoring
• Contact Centers
Classification – route mails to appropriate team
DALL·E 2 •
• Sentiment – prioritize angry customers
• Creative ideation
• Entity extraction and search – analyze liability and risk
• Podcast and music playlist images
• Mail and call transcript summarization
• Content syndication
• Customer response email generation
• Marketing campaign personalization
• Rapid response marketing campaigns: classification, sentiment, summarization,
• Hyper-personalization
content generation, image generation
 Sample playground prompts
• Your identity was stolen, write a letter to Experian asking to remove bad account from your credit report. Mention the right laws

• I want you to act as a research proposal writer. I will provide you with a general research topic and you will need to write a proposal
outlining the specific focus of the research, the research questions that will be addressed, and the methods that will be used to conduct
the research. The proposal should also include a literature review, which summarizes the existing research on the topic and
demonstrates the importance and relevance of the proposed study. The proposal should be well-written and should clearly convey the
value and potential contributions of the proposed research. Topic: how global warming affects human?

• Decide whether a Text's sentiment is Positive, Neutral, or Negative.

Text: "All the training you have done with Rico has been very helpful. You're giving him a great start to his internship. I have taken
notice of your leadership skills and will keep this in mind for future projects."
Sentiment:
 How Azure Open AI Service Process Data?

• Abuse monitoring
• Content Filtering
Common
GovernanceQuestions
and Controls
 Document Process Automation
Extract rich insights from documents and summarizing them

Cosmos DB

PDF OCR
pipeline

Web
Application

Azure Form Azure Cognitive

Documents Recognizer Search

Power BI

Azure OpenAI
Service
GPT Smart Search Use Case
Common Questions
• How to overcome the Model token limits?
• How to centrally govern and control deployment and management
of Azure OpenAI Service?
• How to approach which model to use and if OpenAI is suitable for a
certain use case?
• Can you create multiple deployments for the same model?
• When to use other Cognitive Services offerings, such has Language
models vs OpenAI?
• What does token calculation includes?
• How is customer data handled?
• How content is filtered and can this is disabled?
• What is the difference between prompt engineering vs fine-
tunning?