AWS PARTNER NETWORK

Prompt engineering
for generative AI workloads
Session 1

Aramide Kehinde
Sr. Partner Solution Architect, Data &
AI

© 2024, Amazon©Web
2024,
Services,
Amazon Inc.
Webor Services,
its affiliates.
Inc.All
orrights
its affiliates.
reserved.All rights reserved.
Modules
Module 1: Prompting 101

Module 2: COSTAR Framework

Module 3: Zero Shot Prompting

Module 4: Few Shot Prompting

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 2
Prompting 101

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 3
What is prompting?
What is a Prompt?
Text input provided to an AI system to retrieve a response

What is Prompt Engineering?

Using NLP techniques to craft prompts that steer FMs/LLMs towards desired
responses

Why is this important?

Enables fine-grained and strategic control over models’ behavior
Targets desired capabilities
Mitigates risks

Input Output
Prompt FM Response

*NLP = Natural Language Processing

FM = Foundation Model
LLM = Large Language Model

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why Prompt Engineering?

Cost Expertise
Cost in training/fine No to low expertise is
tuning Large models needed

Open Source Time to Market

Availability of pre trained Faster development
open source models

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Prompt Structure (Most basic)

Act as an IT technical expert providing customer service. Consider the Context below to
Instructions answer the user's questions with a friendly tone. Answer in English in 2 sentences or
less providing instructions.

Context: You work in the Support line of a technology company that commercializes Android
Context smartphones.
The user is calling because the phone is not charging

User input User: Hi, how can I fix my phone?

Assistant:
Output Indicator I will try my best to assist you with this. Can you please tell me the model of your phone
and what version of the Android operating system it is running?

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Does the Prompt’s format matter?

“Models are better at using relevant

information that occurs at the very
beginning or end of its input context”

“Performance degrades significantly

when models must access and use
information located in the middle of its
input context”
Source:
Lost in the Middle: How Language Models Use Long
Contexts
https://arxiv.org/pdf/2307.03172.pdf

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Prompt engineering philosophy
Empirical science: always test your prompts & iterate often!

Develop test Create Test Refine your Share polished

use cases your against prompt prompt
prompt cases

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 8
COSTAR Prompt
Engineering

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 9
COSTAR Prompt Engineering

W H AT I S C O S TA R ?
Context (C)

Objective (O)

Style (S)

Tone (T)

Audience (A)

Response (R)

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 10
COSTAR Prompt Engineering

CONTEXT (C)
Definition: Providing background information helps the foundation model (FM) understand the specific
scenario, ensuring relevance in its responses.

I am a personal productivity developer. In the realm of personal

development and productivity, there is a growing demand for systems that
not only help individuals set goals but also convert those goals into
actionable steps. Many struggle with the transition from aspirations to
concrete actions, highlighting the need for an effective goal-to-system
conversion process.

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 11
COSTAR Prompt Engineering

OBJECTIVE (O)
Definition: Clearly defining the task directs the LLM’s focus to meet that specific goal.

Your task is to guide me in creating a comprehensive system

converter. This involves breaking down the process into distinct steps,
including identifying the goal, employing the 5 Whys technique, learning
core actions, setting intentions, and conducting periodic reviews. The aim
is to provide a step-by-step guide for seamlessly transforming goals into
actionable plans.

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 12
COSTAR Prompt Engineering

STYLE (S)
Definition: Specifying the desired writing style, such as emulating a famous personality or professional
expert, guides the LLM to align its response with your needs.

Write in an informative and instructional style, resembling a guide

on personal development. Ensure clarity and coherence in the
presentation of each step, catering to an audience keen on enhancing
their productivity and goal attainment skills.

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 13
COSTAR Prompt Engineering

TONE (T)
Definition: Setting the tone ensures the response resonates with the required sentiment, whether it be
formal, humorous, or empathetic.

Maintain a positive and motivational tone throughout, fostering a

sense of empowerment and encouragement. It should feel like a friendly
guide offering valuable insights.

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 14
COSTAR Prompt Engineering

AUDIENCE (A)
Definition: Identifying the intended audience tailors the LLM’s response to be appropriate and
understandable for specific groups, such as experts or beginners.

The target audience is individuals interested in personal

development and productivity enhancement. Assume a readership
that seeks practical advice and actionable steps to turn their goals into
tangible outcomes.

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 15
COSTAR Prompt Engineering

RESPONSE (R)
Definition: Providing the response format, like a list or JSON, ensures the LLM outputs in the required
structure for downstream tasks.

Provide a structured list of steps for the goal-to-system

conversion process. Each step should be clearly defined, and the overall
format should be easy to follow for quick implementation.

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 16
Zero-shot prompting

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 20
Zero-shot prompting
A zero-shot prompt is a prompt that does not include any examples of the desired
response.

The prompt should communicate the expected COSTAR elements of the response.

Recommended uses:
• Comparatively long inputs and outputs, where providing examples could be cost-
prohibitive or response time-prohibitive
• Less precise output structure required, or structure can be reliably generated
without providing examples.

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Zero-shot prompting

S E N T I M E N T A N A LY S I S

I thought it was pretty decent.

What is the sentiment of the above text? Respond with either "Positive" or "Negative":

Titan Text G1 - Express Llama 2 Chat 70B

Positive Positive

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 22
Zero-shot prompting

S E N T I M E N T A N A LY S I S
It was meh.

What is the sentiment of the above text? Respond with either "Positive" or "Negative":

Titan Text G1 - Express Llama 2 Chat 70B

Negative Negative

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 23
Few-shot prompting

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 24
Few-shot prompting
A few-shot prompt is a prompt that includes examples of the desired response.
This can help the model understand exactly what a “good” response looks like.

The examples might indicate the appropriate COSTAR elements of the response.

Each large language model may have distinct approaches for how the examples
should be included in the prompt.

Recommended uses:
• Comparatively short inputs or outputs, (try to minimize excess token
consumption)
• Precise output structure required, which could be challenging to achieve with
zero-shot prompting

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Few-shot prompting
S E N T I M E N T A N A LY S I S

Titan Text G1 - Express Cohere Command

I liked it //Positive This is a review sentiment classifier.

It was OK //Positive Review: "I liked it"
Couldn't believe how bad it Sentiment: Positive
was!//Negative Review: "It was OK"
Not my favorite //Negative Sentiment: Positive
This is meh. Review: "Couldn't believe how bad it
What is the sentiment of the above text? was!"
Sentiment: Negative
Review: "Not my favorite"
Sentiment: Negative
Review: "This is meh."
Sentiment:
Negative Negative

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 26
Notes on newer
Anthropic models

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 27
 NEW!
Anthropic Claude Messages API
Advantages
Claude 3 Sonnet

“system”: “You’re an IT expert. Always respond

 No “Human” & “Assistant” tags
with simple and direct answers in a single required
sentence.”,
“messages”: [
 Adopting universal format,
{ compatible with other LLMs and
“role”: “user”, open-source tools
“content”: “How can I reboot my Mac laptop?”
},
{  Support for other content types
“role”: “assistant”, (e.g. images) as part of the inputs
“content”: “To reboot your Mac laptop,”
}
 Combination of performance and
] latency of previous Claude models

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Text Completion vs Messages APIs for prompting
Bedrock APIs – Invoking Claude with parameters (boto3)

NEW!
Claude Text Completions API Claude Messages API

{ {
"prompt": "\n\nHuman:<prompt>\n\nAssistant:", "anthropic_version": "bedrock-2023-05-31",
"temperature": float, "max_tokens": int,
"top_p": float, "system": "<system_prompt>", <- Optional
"top_k": int, "messages": [{
"max_tokens_to_sample": int, "role": "<role>",
"stop_sequences": [string] "content": "<prompt>"
} }],
"temperature": float,
"top_p": float,
"top_k": int,
"stop_sequences": [string]
}

Note: You MUST use Messages API for invoking Claude 3 Sonnet

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Anthropic Claude 3
Example

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 30
Anthropic Claude Text Completions API

system prompt

Claude Instant or Claude 2.0

“prompt”: “\n\nHuman: You’re an IT expert. Always respond with direct answers in a single sentence.\nHow
can I reboot my Mac laptop?\nAssistant: To reboot your Mac laptop,”

user prompt assistant pre-filling

system prompt

Claude 2.1 (with system prompt)

“prompt”: “\n\nYou’re an IT expert. Always respond with direct answers in a single sentence.\n\nHuman:
How can I reboot my Mac laptop?\nAssistant: To reboot your Mac laptop,”

user prompt assistant pre-filling

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 32
messages = [ { "role": 'user', "content": [ {"image": { "format": 'jpeg',
"source": {"bytes":
get_base64_encoded_image("./images/best_practices/nine_dogs.jpg")} }, },
{"text": "You have perfect vision and pay great attention to detail which makes
you an expert at counting objects in images. How many dogs are in this picture?
Before providing the answer in <answer> tags, think step by step in <thinking>
tags and analyze every part of the image."}, ] } ] converse_api_params =
{ "modelId": modelId, "messages": messages, } response =
bedrock_client.converse(**converse_api_params) print(response['output']
['message']['content'][0]['text'])

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 33
 Thank you!

© 2024, Amazon©Web
2024,
Services,
Amazon Inc.
Webor Services,
its affiliates.
Inc.All
orrights
its affiliates.
reserved.All rights reserved.
Anthropic Claude Text Completions API

Claude Instant or Claude 2.0

“prompt”: “\n\nHuman: You’re an IT expert. Always respond with direct answers in a single sentence.\nHow
can I reboot my Mac laptop?\nAssistant: To reboot your Mac laptop,”

Claude 2.1 (with system prompt)

“prompt”: “\n\nYou’re an IT expert. Always respond with direct answers in a single sentence.\n\nHuman:
How can I reboot my Mac laptop?\nAssistant: To reboot your Mac laptop,”

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Anthropic Claude Messages API - Text
Single user example

“messages”: [
{“role”: “user”, “content”: “Hello Claude!”}
]

Multi-turn conversation example

“messages”: [
{“role”: “user”, “content”: “Hello there”},
{“role”: “assistant”, “content”: “Hi, I’m Claude. How can I help you?”},
{“role”: “user”, “content”: “Can you explain LLMs in plain English?”}
]

Partially-filled response example

“messages”: [
{“role”: “user”, “content”: “Please describe yourself using only JSON”},
{“role”: “assistant”, “content”: “Here is my JSON description\n{”}
]

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.