Prompt Engineering for Generative AI

Generative AI and Prompt Engineering

Ram N Sangwan
Prompt – LLMs as Next Word Predictors

Text prompts are

how users interact
with LLMs
LLM attempt to produce
the next series of words
that are most likely to
follow from the previous
text

Prompt Completion
Four score and seven years ago Fathers brought forth on this continent a new nation ..
our
Prompt & Prompt Engineering

Prompt
• The input or initial text provided
to the model.
Prompt Engineering
• The process of iteratively refining
a prompt for the purpose of
eliciting a particular style of
response.
Aligning LLMs to Follow Instructions
Completion LLMs are trained to predict the next word on a large dataset of
internet text, rather than to safely perform the language task that the user
wants.
Can not give instructions or ask questions to a completion LLM.

Instead, formulate your input as a prompt whose natural continuation is your

desired output.
Prompt Instructions
Goal

• Ensure that the language model understands the user’s intent correctly and provides accurate
responses.
In-Context Learning and K-Shot Prompting
In-context learning – Conditioning (Prompting) an LLM with instructions and/or demonstrations
of the task it is meant to complete.
K-Shot Prompting – Explicitly providing k examples of the intended task in the prompt.

Translate English to French: task description

sea otter => loutre de mer examples
peppermint => menthe poivrée
plush giraffe => girafe peluche
cheese => prompt

Few-shot prompting is widely believed to improve results over 0-shot prompting

K-Shot Inference

• No example
Zero Shot • The goal is for the Model to make predictions about
new classes by using prior knowledge.

• One example
One Shot • The goal is to make predictions for the new classes
based on this single example

• Some examples
Few Shot • The goal is to make predictions for new classes
based on few examples of labeled data.
 Zero Shot Example
• Let's say you want to use an LLM for translation without any fine-tuning or training.
• You can provide the model with a zero-shot prompt like this:

Prompt: "Translate the following English text to French: 'Hello, how are you?’”

• The model understands the structure and semantics of languages and can generate a reasonable
translation in French:

Generated Response: "Bonjour, comment ça va ?"

 One Shot Example
• Let's consider a model that has never been trained to generate recipes. With one-shot prompting,
you provide the model with a single example recipe:

Prompt: "Generate a recipe for chocolate chip cookies."

Example Recipe: "Ingredients: butter, sugar, eggs, flour, chocolate chips.
Instructions: Preheat oven to 350°F. Mix butter and sugar..."

• Even without specific example, it can use the structure of the provided example to generate a new recipe:

Generated Recipe: "Ingredients: margarine, brown sugar, egg substitute, all-purpose

gluten-free flour, dairy-free chocolate chips.
Instructions: Preheat oven to 350°F. Cream margarine and brown sugar..."
Prompt Formats
LLMs are trained on a specific prompt format. If you format prompts in a different way, you
may get odd/inferior results.
Llama-3 Prompt Formatting:
Beginning of the
<<s>> Instructions
Beginning of the
entire sequence. [INST]
<<SYS>>
System prompt to set {{system_prompt}}
model context User Message specifying
<</SYS>> instructions to the model
{{user_message}}
[/INST]
End of the Instruction
Other Angles to add Specificity to a Prompt

Style Persona Length

Tone
Telling the model to
provide a response Adding how the tone of a Telling the model to act
that follows a certain piece of text should be. like a certain persona. Telling the model to
style or framework. E.g, “Tone: casual” E.g, “You are a world-class generate text with a specific
content marketer. Write a length, in words,
E.g., product description for…” paragraphs, and others.
“Generate an ad copy for a wireless
headphone product” E.g, “Write in three paragraphs
V/s the benefits of …”
“Generate an ad copy for a wireless
headphone product, following the
AIDA Framework – Attention,
Interest, Desire, Action.”
 Best Practices

Write prompt in multiple ways

Try writing the prompt in multiple ways till you get best
response.
Describe
Describe the task with clear and specific instructions

Handle Edge Cases

Handle edge cases and conditions response

Give Examples
Give Examples for completing the task for a desirable outcome
Tokens

• Language models understand “token” rather than characters.

• One token can be a part of a word, an entire word, or punctuation.
• A Common word such as “apple” is a token.
• A word such as “friendship” is made up of two tokens – “friend” and
“ship”.
• Number of Tokens/word depend on the complexity of the text.
• Simple text: 1 token/word (Avg.)
• Complex text (less common words): 2-3 tokens/word (Avg.)
Max Tokens

• This is the maximum length of the output that the model

can generate in one response, measured in tokens.
• If the max tokens limit is set, the model will not generate
more tokens than that limit in its response.
Temperature
Temperature is a (hyper) parameter that controls the randomness of the LLM output
Temperature modulates the distribution over vocabulary

The sky is ________

Temperature of 0 makes the model deterministic (limits the model to use the word with the highest probability).
When temperature is increased, the distribution is flattened over all words.
With increased temperature, model uses words with lower probabilities
Top k

Top k tells the model to pick the next token from top ”k” tokens in its list, sorted by
probability.

Name of that country is the ______________

If Top k is set to 3, Model will only pickup from

the Top 3 options and ignore all others.

Mostly pick “United”. But may pick

“Netherlands” and “Czech” at times.
Top p

Top p is similar to “Top k” but picks from the top tokens based on the sum of their
probabilities.

Name of that country is the ______________

If p is set to .15, then it will only pickup from

United and Netherlands as their probabilities
add up to 14.7%.
Stop Sequences
• Is a string that tells the model to stop generating more content.
• It is a way to control your model output.
• If a period (.) is used as a stop sequence, the model stops generating text once
it reaches the end of the first sentence, even if the number of token limit is
much higher.
• Examples in Lab 13 Day 4.
 Frequency and Presence Penalties

These are useful if you want to get rid of repetition in your outputs.

Frequency Penalty penalizes tokens that have already appeared in the

preceding text (including the prompt), and scales based on how many
times that token has appeared.
A Token that has already appeared 10 times gets a higher penalty (which
reduces its probability of appearing) than a token that has appeared only
once.
Presence penalty applies the penalty, regardless of frequency. As long as
the token has appeared once before, it will get penalized.
Chain of Thought (CoT)
(b) Few-shot-CoT
(a) Few Shot
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?
can has 3 tennis balls. How many tennis balls does he have now?
A: Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis
A: The answer is 11. balls. 5 + 6 = 11. The answer is 11.
Q: A juggler can juggle 16 balls. Half of the balls are golf balls, and Q: A juggler can juggle 16 balls. Half of the balls are golf balls, and
half of the golf balls are blue. How many blue golf balls are there? half of the golf balls are blue. How many blue golf balls are there?
A: A:
(Output) The answer is 8. X (Output) The juggler can juggle 16 balls. Half of the balls are golf
balls. So there are 16 / 2 = 8 golf balls. Half of the golf balls are blue.
So there are 8 / 2 = 4 blue golf balls. The answer is 4. ✔

(c) Zero-shot (d) Zero-shot-CoT (Ours)

Q: A juggler can juggle 16 balls. Half of the balls are golf balls, and Q: A juggler can juggle 16 balls. Half of the balls are golf balls, and
half of the golf balls are blue. How many blue golf balls are there? half of the golf balls are blue. How many blue golf balls are there?
A: The answer (arabic numerals) is A: Let's think step by step.
(Output) 8 X (Output) There are 16 balls in total. Half of the balls are golf balls.
That means that there are 8 golf balls. Half of the golf balls are
blue. That means that there are 4 blue golf balls.
Chain of Thought – Use and Not to Use Cases
Use Cases Not to Use Cases
Complex Problem Solving: Simple Queries:
• Mathematical Problems. • Fact Retrieval.
• Logical Reasoning. • Yes/No Questions.
• Decision Making. Limited Context:
Detailed Explanations: • Brief Responses.
• Learning and Education. • Just-in-Time Information.
• Language Translation. Tasks Requiring High Efficiency:
Step-by-Step Instructions: • Real-time Applications.
• Technical Support. • Resource-Intensive Environments.
• Processes and Workflows. Repetitive or Predictable Responses:
Expanding Creativity: • Common Queries.
• Storytelling: • Standard Procedures.
• Brainstorming Ideas. Privacy and Security Sensitive Information:
Clarifying Ambiguities: • Sensitive Data Processing.
• Legal and Ethical Reasoning. • Anonymity and Confidentiality
• Philosophical Discussions.
 lamp_review = """

Inferring Prompts Needed a nice lamp for my bedroom, and this one had \
additional storage and not too high of a price point. \
Got it fast. The string to our lamp broke during the \
transit and the company happily sent over a new one. \
Came within a few days as well. It was easy to put \
together. I had a missing part, so I contacted their \
support and they very quickly got me the missing piece! \
Lumina seems to me to be a great company that cares \
about their customers and products!!
"""

prompt = f"""
What is the sentiment of the following product review,
which is delimited with triple backticks?

Review text: ```{lamp_review}```

"""
response = get_completion(prompt)
print(response)
Translation
 prod_review = """

Summarizing Got this panda plush toy for my daughter's birthday, \

who loves it and takes it everywhere. It's soft and \
super cute, and its face has a friendly look. It's \
a bit small for what I paid though. I think there \
might be other options that are bigger for the \
same price. It arrived a day earlier than expected, \
so I got to play with it myself before I gave it \
to her.
"""
prompt = f"""
Your task is to generate a short summary of a product \
review from an ecommerce site.

Summarize the review below, delimited by triple

backticks, in at most 30 words.

Review: ```{prod_review}```
"""

response = get_completion(prompt)
print(response)
Thank You