Introduction

• To build a support application for text summarization and code suggestions,

you can leverage ChatGPT, a chatbot by OpenAI, which generates humanlike
responses from natural language input.

• ChatGPT is powered by generative AI models, a subset of deep learning

algorithms, capable of handling tasks across vision, speech, language,
decision, search, and more.

• Azure OpenAI Service integrates these generative AI models into the Azure
platform, offering security, scalability, and integration with other Azure
services.

• This allows you to create advanced AI solutions.

• The models can be accessed via REST API, various SDKs, and the Azure OpenAI
Studio interface.

• The module provides guidance on using the Azure OpenAI Studio, laying the
groundwork for developing applications utilizing generative AI.
Access Azure OpenAI
Service
• The first step in building a generative AI solution with Azure OpenAI is to
provision an Azure OpenAI resource in your Azure subscription.

• Azure OpenAI Service is currently in limited access. Users need to apply for
service access at https://aka.ms/oai/access.

• Once you have access to Azure OpenAI Service, you can get started by
creating a resource in the Azure portal or with the Azure command line
interface (CLI).
Access Azure OpenAI
Service
Create an Azure OpenAI Service resource in the
Azure portal
• When you create an Azure OpenAI
Service resource, you need to
provide a subscription name,
resource group name, region,
unique instance name, and select
a pricing tier.
Access Azure OpenAI
Service
Create an Azure OpenAI Service resource in Azure CLI

• To create an Azure OpenAI Service resource from the CLI, refer to this example
and replace the following variables with your own:

• MyOpenAIResource: replace with a unique name for your resource

• OAIResourceGroup: replace with your resource group name
• eastus: replace with the region to deploy your resource
• subscriptionID: replace with your subscription ID

az cognitiveservices account create \

n MyOpenAIResource \
g OAIResourceGroup \
l eastus \
kind OpenAI \
sku s0 \
subscription subscriptionID
Access Azure OpenAI
Service availability
Regional

• Azure OpenAI Service provides access to many types of models.

• Certain models are only available in select regions.

• Consult the Azure OpenAI model availability guide for region availability.

• You can create two Azure OpenAI resources per region.

Use Azure OpenAI
Studio
• Azure OpenAI Studio provides access to model management, deployment,
experimentation, customization, and learning resources.

• You can access the Azure OpenAI Studio through the Azure portal after creating a
resource, or at https://oai.azure.com by logging in with your Azure OpenAI
resource instance.

• During the signin workflow, select the appropriate directory, Azure subscription,
and Azure OpenAI resource.
Use Azure OpenAI
Studio
Use Azure OpenAI
Studio
• When you first open Azure OpenAI
Studio, you'll see a calltoaction
button at the top of the screen to
deploy your first model.

• Selecting the option to create a

new deployment opens
the Deployments page, from
where you can deploy a base
model and start experimenting
with it.
Explore types of generative AI models
• To begin building with Azure OpenAI, you need to choose a base model and deploy
it.

• Microsoft provides base models and the option to create customized base models.
This module covers the currently available base models.

• Azure OpenAI includes several types of model:

• GPT4 models are the latest generation of generative pretrained (GPT) models
that can generate natural language and code completions based on natural
language prompts.

• GPT 3.5 models can generate natural language and code completions based
on natural language prompts. In particular, GPT35turbo models are
optimized for chatbased interactions and work well in most generative AI
scenarios.

• Embeddings models convert text into numeric vectors, and are useful in
language analytics scenarios such as comparing text sources for similarities.

• DALLE models are used to generate images based on natural language

prompts. Currently, DALLE models are in preview. DALLE models aren't listed
Explore types of generative AI models
• Models differ by speed, cost, and how
well they complete specific tasks.

• You can learn more about the

differences and latest models offered
in the
Azure OpenAI Service documentation.

• In the Azure OpenAI Studio,

the Models page lists the available
base models (other than DALLE
models) and provides an option to
create additional customized models
by finetuning the base models.

• The models that have

a Succeeded status mean they're
successfully trained and can be
selected for deployment.
Deploy generative AI models
• You first need to deploy a model to make API calls to receive completions to
prompts.

• When you create a new deployment, you need to indicate which base model to
deploy.

• You can deploy any number of deployments in one or multiple Azure OpenAI
resources as long as their TPM adds up to less than 240K total in that region.

•Deploy
There areusing
several Azure
ways youOpenAI
can deploy Studio
your base model.

• In Azure OpenAI Studio's Deployments page, you can create a new deployment
by selecting a model name from the menu.

• The available base models come from the list in the models page.
Deploy generative AI models
Deploy generative AI models
• From the Deployments page in the Studio, you can also view information about
all your deployments including deployment name, model name, model version,
status, date created, and more.

Deploy using Azure

•CLI
You can also deploy a model using the console.
• Using this example, replace the following variables with your own resource
values:
• OAIResourceGroup: replace with your resource group name
• MyOpenAIResource: replace with your resource name
• MyModel: replace with a unique name for your model
• gpt35turbo: replace with the base model you wish to deploy
az cognitiveservices account deployment create g OAIResourceGroup n
MyOpenAIResource \
deploymentname MyModel \
modelname gpt35turbo \
modelversion "0301" \
modelformat OpenAI \
skuname "Standard" \
skucapacity 1
Deploy generative AI models
Deploy using the REST
API
• You can deploy a model using the REST API.

• In the request body, you specify the base model you wish to deploy
Use prompts to get completions from
models
• Once the model is deployed, you can test how it completes prompts.

• A prompt is the text portion of a request that is sent to the deployed model's
completions endpoint.

• Responses are referred to as completions, which can come in form of text, code,
or other formats.
Use prompts to get completions from
models
Prompt types
Task type Prompt example Completion
example
Classifying content Tweet: I enjoyed the trip. Positive
Sentiment:
Generating new content List ways of traveling 1. Bike
2. Car ...
Holding a conversation A friendly AI assistant See examples
Transformation (translat English: Hello bonjour
ion and symbol French:
conversion)
Summarizing content Provide a summary of the content The content shares
{text} methods of machine
learning.
Picking up where you One way to grow tomatoes is to plant seeds.
left off
Giving factual How many moons does Earth One
responses have?
Use prompts to get completions from
models
Completion quality
• Several factors affect the quality of completions you'll get from a generative AI
solution.

• The way a prompt is engineered.

• The model parameters (covered next)

• The data the model is trained on, which can be adapted through model
finetuning with customization

• You have more control over the completions returned by training a custom
model than through prompt engineering and parameter adjustment.
Use prompts to get completions from
models
Making calls
• You can start making calls to your deployed model via the REST API, Python,
C#, or from the Studio.

• If your deployed model has a GPT3.5 or GPT4 model base, use the Chat
completions documentation, which has different request endpoints and
variables required than for other base models.
 Test models in Azure OpenAI Studio's
• playgrounds
Playgrounds are useful interfaces in Azure OpenAI Studio that you can use to
experiment with your deployed models without needing to develop your own
client application.

• Azure OpenAI Studio offers multiple playgrounds with different parameter

tuning options.
Completions
playground
• The Completions playground allows you to make calls to your deployed
models through a textin, textout interface and to adjust parameters.

• You need to select the deployment name of your model under Deployments.

• Optionally, you can use the provided examples to get you started, and then
you can enter your own prompts.
Test models in Azure OpenAI Studio's
playgrounds
Test models in Azure OpenAI Studio's
playgrounds
Completions Playground
parameters
• There are many parameters that you can adjust to change the performance of
your model:

• Temperature: Controls randomness

o Lower values: more deterministic responses
o Higher values: more creative responses
o Adjust temperature or Top P, not both

• Max length (tokens): Limits tokens per response

o 4000-token maximum shared between prompt and response
o One token is roughly four characters in typical English text

• Stop sequences: Halts responses at desired points

o Specify up to four sequences
Test models in Azure OpenAI Studio's
playgrounds
Completions Playground
parameters
• Top probabilities (Top P): Controls randomness
o Lowering Top P: narrows token selection to likely tokens
o Increasing Top P: broadens selection to include less likely tokens
o Adjust temperature or Top P, not both

• Frequency penalty: Reduces repeated tokens proportionally

o Decreases repeated text

• Presence penalty: Reduces any repeated token

o Increases new topics

• Preresponse text: Insert text after user input

o Prepares the model for a response

• Postresponse text: Insert text after model's response

o Encourages further user input
Test models in Azure OpenAI Studio's
playgrounds
Chat playground
• The Chat playground is based on a conversation-in, message-out interface.

• You can initialize the session with a system message to set up the chat
context.

• In the Chat playground, you're able to add few-shot examples.

• The term few-shot refers to providing a few of examples to help the model
learn what it needs to do.

• You can think of it in contrast to zero-shot, which refers to providing no

examples.

• In the Assistant setup, you can provide few-shot examples of what the user
input may be, and what the assistant response should be.

• The assistant tries to mimic the responses you include here in tone, rules, and
format you've defined in your system message.
Test models in Azure OpenAI Studio's
playgrounds
Chat playground
Test models in Azure OpenAI Studio's
playgrounds
Chat playground parameters
• The Chat playground, like the Completions playground, also includes the
Temperature parameter.
• The Chat playground also supports other parameters not available in the
Completions playground. These include:

• Max response: Set a limit on the number of tokens per model response. The
API supports a maximum of 4000 tokens shared between the prompt
(including system message, examples, message history, and user query) and
the model's response. One token is roughly four characters for typical English
text.

• Top P: Similar to temperature, this controls randomness but uses a different

method. Lowering Top P narrows the model’s token selection to likelier tokens.
Increasing Top P lets the model choose from tokens with both high and low
likelihood. Try adjusting temperature or Top P but not both.

• Past messages included: Select the number of past messages to include in

each new API request. Including past messages helps give the model context
for new user queries. Setting this number to 10 will include five user queries
and five system responses.
Test models in Azure OpenAI Studio's
playgrounds
Chat playground parameters
• The Current token count is viewable from the Chat playground.

• Since the API calls are priced by token and it's possible to set a max response
token limit, you'll want to keep an eye out for the current token count to make
sure the conversation-in doesn't exceed the max response token count.
Exercise - Get started with Azure OpenAI
Service
 Knowledge check
1. What Azure OpenAI base model can you deploy to access the capabilities of ChatGPT?

a) text-davinci-003
b) gpt-35-turbo
c) text-embedding-ada-002 (Version 2)
2. Which parameter could you adjust to change the randomness or creativeness of the
completions returned?

d) Temperature
e) Frequency penalty
f) Stop sequence
3. Which Azure OpenAI Studio playground is able to support conversation-in, message-out
scenarios?

g) Completions
h) Chat
i) Bot