L3:Prompt Engineering

Ranjitha P-20213CSE0014 1
Prompt Engineering-Introduction, prompt design and prompt engineering
https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/introduction-prompt-design

• A prompt is a natural language request submitted to a language model to receive

a response back.
• They can contain questions, instructions, contextual information, few-shot
examples, and partial input for the model to complete or continue.
• depending on the type of model being used, it can generate text, embeddings,
code, images, videos, music, and more.
• Prompt design is the process of creating prompts that elicit the desired response
from language models.
• Writing well structured prompts can be an essential part of ensuring accurate,
high quality responses from a language model.

Ranjitha P-20213CSE0014 2
Prompt Engineering-Introduction
https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/introduction-prompt-design

• The iterative process of repeatedly updating prompts and assessing the model's
responses is sometimes called prompt engineering.
• for straightforward tasks many of the models (Google’s Gemini) often perform
well without the need for prompt engineering.
• for complex tasks, effective prompt engineering still plays an important role.

Ranjitha P-20213CSE0014 3
Components of a Prompt
• Prompt contents fall within one of the following components:
1. Task (required)
2. System instructions (optional)
3. Few-shot examples (optional)
4. Contextual information (optional)
1. Task:
• A task is the text in the prompt that you want the model to provide a
response for.
• Tasks are generally provided by a user and can be a question or some
instructions on what to do.

Ranjitha P-20213CSE0014 4
Components of a Prompt..
• Example of a task with a question:

Ranjitha P-20213CSE0014 5
Components of a Prompt..
• Example of a task with an instruction:

Ranjitha P-20213CSE0014 6
Components of a Prompt..
2. System Instructions:
• Are instructions passed to the model before user input.
• They dictate the style and tone of the model – what the model should and
should not do
• They are given through : “system parameter”

Ranjitha P-20213CSE0014 7
Ranjitha P-20213CSE0014 8
3. Few shot examples:
• Few-shot examples are examples that you include in a prompt to show the model what getting it
right looks like.
• Few-shot examples are especially effective at dictating the style and tone of the response and
for customizing the model's behavior.

Ranjitha P-20213CSE0014 9
4. Contextual Information:
• Contextual information, or context, is information that is included in the prompt that the
model uses or references when generating a response.
• We can include contextual information in different formats, like tables or text.

Ranjitha P-20213CSE0014 10
Safety and Fallback responses

• In few use cases the model is not expected to fulfill the user's requests.
• when the prompt is encouraging a response that is not aligned with the service provider’s
(Google's/OpenAI’s) values or policies, the model might refuse to respond and provide a fallback
response.
• Few cases where the model is likely to refuse to respond:
• Hate Speech:
• Prompts with negative or harmful content targeting identity and/or protected attributes.
• Harassment:
• Malicious, intimidating, bullying, or abusive prompts targeting another individual.
• Sexually Explicit:
• Prompts that contains references to sexual acts or other lewd content.
• Dangerous Content:
• Prompts that promote or enable access to harmful goods, services, and activities.

Ranjitha P-20213CSE0014 11
Prompting Strategies
https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/prompt-design-strategies

• There is no specific syntax/format of a prompt

• There are common strategies that can be used to affect the model's responses.
• Rigorous testing and evaluation remain crucial for optimizing model performance.
• Large language models (LLM) are trained on vast amounts of text data to learn
the patterns and relationships between units of language.
• When given some text (the prompt), language models can predict what is likely to
come next, like a sophisticated autocompletion tool.
• Therefore, when designing prompts, consider the different factors that can
influence what a model predicts comes next.

Ranjitha P-20213CSE0014 12
Prompt Engineering Workflow…
• Prompt engineering is a test-driven and iterative process that can
enhance model performance.
• When creating prompts, it is important to clearly define the objectives
and expected outcomes for each prompt and systematically test them
to identify areas of improvement.

Ranjitha P-20213CSE0014 13
Prompt Engineering Workflow…
• Two factors affecting the effectiveness of a prompt : content and
structure.
• Content:
• has all relevant information associated with a task for the model to complete
the task.
• It can include instructions, examples, contextual information, and so on.
• Structure:
• Helps to give the relevant information in a structure, easy to parse for the
model.
• Structure of the prompt is defined by prompt template
• Things like the ordering, labeling, and the use of delimiters can all affect the
quality of responses.

Ranjitha P-20213CSE0014 14
Ranjitha P-20213CSE0014 15
Ranjitha P-20213CSE0014 16
Best Practices of Prompt design
• Give clear and specific instructions
• Include few-shot examples
• Assign a role
• Add contextual information
• Use system instructions
• Structure prompts
• Instruct the model to explain its reasoning
• Break down complex tasks
• Experiment with parameter values
• Prompt iteration strategies

Ranjitha P-20213CSE0014 17
1. Give clear and specific instructions
• Use Effective instructions applying the following principles:
• Tell the model what to do.
• Be clear and specific.
• Specify any constraints or formatting requirements for the output.
• Example: Suppose you own a cheeseburger restaurant and you want
to use a model to help you learn about which menu items are the
most popular. You want the model to format transcripts of customer
orders in JSON so that you can quickly identify menu items

Ranjitha P-20213CSE0014 18
Case 1: Response if the instruction is more generic

Ranjitha P-20213CSE0014 19
Ranjitha P-20213CSE0014 20
Case 2: Response if the instruction is specific following the principles

Ranjitha P-20213CSE0014 21
2. Include few-shot examples
• include examples in the prompt that show the model what a good response looks
like.
• The model attempts to identify patterns and relationships from the examples and
applies them when generating a response.
• Prompts that contain examples are called few-shot prompts, while prompts that
provide no examples are called zero-shot prompts.
• Few-shot prompts are often used to regulate the output formatting, phrasing,
scoping, or general patterning of model responses.
• Use specific and varied examples to help the model narrow its focus and generate
more accurate results.
• Experiment with the number of prompts to include. Depending on the model, too
few examples are ineffective at changing model behavior. Too many examples can
cause the model to overfit.

Ranjitha P-20213CSE0014 22
Zero-shot versus few-shot prompts
Extract the technical specifications from the text given and give the
output in JSON
Case 1: Response from a zero-shot prompt

Ranjitha P-20213CSE0014 23
Case 2: If we want the keys in lower case, use few-shot prompt

Ranjitha P-20213CSE0014 24
3. Assign a role

• Assign a role to the model (in persona)

• Adding a role is not always necessary but can enforce a certain level
of expertise when generating a response, improve performance, and
tailor its communication style.
• particularly useful for getting the model to perform highly technical
tasks or enforcing specific communication styles.

Ranjitha P-20213CSE0014 25
Ranjitha P-20213CSE0014 26
4. Add contextual information
• instead of assuming that the model has all of the required
information, can include instructions and information that the model
needs to solve a problem, in the prompt.
• This contextual information helps the model understand the
constraints and details of what it is asked to do.
• Effective contextual information includes the following:
• Background information (context) for the model to refer to when generating
responses.
• Rules or pre-programmed responses to steer the model behavior.
Ex: Ask the model to give troubleshooting guidance for a router:
https://
cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/contextual-infor
mation

Ranjitha P-20213CSE0014 27
5. System Instructions
• System instructions are like a preamble added before the LLM gets
exposed to any further instructions from the user.
• It lets users steer the behavior of the model based on their specific
needs and use cases.
• When you set a system instruction, you give the model additional
context to understand the task, provide more customized responses,
and adhere to specific guidelines over the full user interaction with
the model.
• Example, include things like the role or persona, contextual
information, and formatting instructions:

Ranjitha P-20213CSE0014 28
https://
cloud.google.com/vertex-ai/generative-ai/docs/learn/prom
pts/system-instructions

Ranjitha P-20213CSE0014 29
6. Structure prompts [Same as discussed before using prompt
templates]
7. Instruct the model to explain its reasoning
8. Break down complex tasks
9. Experiment with parameter values
10. Prompt iteration strategies

Ranjitha P-20213CSE0014 30
7. Instruct the model to explain reasoning
• the model responds with the steps that it employs to solve the
problem – reasoning steps
• Going through this process can sometimes improve accuracy and
nuance, especially for challenging queries.
• Reasoning steps are part of the response.
• To parse out the reasoning steps from the answer,specify an output
format by using XML or other separators.

Ranjitha P-20213CSE0014 31
• Example, suppose you're writing a cooking blog and you want the
model to tell you how readers might interpret different parts of the
blog. If you don't instruct the model to explain its reasoning, the
response from the model might not be as useful as you'd like:
• https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts
/explain-reasoning

Ranjitha P-20213CSE0014 32
Ranjitha P-20213CSE0014 33
8. Break down complex tasks
• For complex tasks that require multiple instructions or steps, improve
the model's responses by breaking the prompts into subtasks.
• Smaller prompts can help improve controllability, debugging, and
accuracy.
• There are two ways to break down complex prompts and ingest them
into a model:
• Chain prompts: split a task into subtasks(sequential steps) and run the
subtasks sequentially.
• Aggregate responses: split a task into subtasks and run the subtasks in
parallel.

Ranjitha P-20213CSE0014 34
Chain Prompts

• For complex tasks that involve multiple sequential steps, make each
step a prompt and chain the prompts together in a sequence.
• In this sequential chain of prompts, the output of one prompt in the
sequence becomes the input of the next prompt.
• The output of the last prompt in the sequence is the final output.
• For example, suppose you run a telecommunications business and
want to use a model to help you analyze customer feedback to
identify common customer issues, classify issues into categories, and
generate solutions for categories of issues.

Ranjitha P-20213CSE0014 35
Task 1: identify customer issues

Ranjitha P-20213CSE0014 36
Task 2: classify issues into
categories

Ranjitha P-20213CSE0014 37
Task 3: generate solutions

Ranjitha P-20213CSE0014 38
Aggregate responses
• Used for a complex task, that doesn’t follow a sequence.
• you can run parallel prompts and aggregate the model's responses.
• Ex: https://
cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/break-
down-prompts
• Problem: Suppose you own a music record store and want to use a
model to help you decide which records to stock based on music
streaming trends and your store's sales data.
• Analyse the two data sets streaming and sales data in parallel by
running two prompts.

Ranjitha P-20213CSE0014 39
Task 1: analyze streaming and sales data in parallel

Ranjitha P-20213CSE0014 40
Task 2 : aggregate the responses

Ranjitha P-20213CSE0014 41
9. Experiment with parameter values
• Each call sent to a model includes parameter values that control how the model
generates a response.
• The model can generate different results for different parameter values.
• Experiment with different parameter values to get the best values for the task.
• The parameters available for different models may differ.
• The most common parameters are the following:
• Max output tokens
• Temperature
• Top-K
• Top-P

Ranjitha P-20213CSE0014 42
• Max output tokens:
• Maximum number of tokens that can be generated in the response.
• A token is approximately four characters.
• 100 tokens correspond to roughly 60-80 words.
• Specify a lower value for shorter responses and a higher value for potentially longer
responses.
• Temperature:
• Temperature controls the degree of randomness in token selection.
• Lower temperatures are good for prompts that require a less open-ended or creative
response, while higher temperatures can lead to more diverse or creative results.
• Temp=0 always selects the tokens with the highest probability.
• Temp=0, always generates deterministic response, with little randomness still possible.
• If the model returns a response that's too generic, too short, or the model gives a fallback
response, try increasing the temperature.

Ranjitha P-20213CSE0014 43
• Top-K:
• Top-K changes how the model selects tokens for output.
• Top-K = 1, the next selected output token is the most probable among all
tokens in the model's vocabulary (also called greedy decoding)
• Top-K = 3, the next output token is selected from among the three most
probable tokens by using temperature.
• For each token selection step, the top-K tokens with the highest probabilities
are sampled.
• Then tokens are further filtered based on top-P with the final token selected
using temperature sampling.
• Specify a lower value for less random responses and a higher value for more
random responses.

Ranjitha P-20213CSE0014 44
• Top-P:
• Top-P changes how the model selects tokens for output
• Tokens are selected from the most propable(see top-K) to least probable until
the sum of their probabilities equals the top-P value.
• For example, if tokens A, B, and C have a probability of 0.3, 0.2, and 0.1 and
the top-P =0.5, then the model will select either A or B as the next token by
using temperature and excludes C as a candidate.
• Specify a lower value for less random responses and a higher value for more
random responses.

Ranjitha P-20213CSE0014 45
10. Prompt Iteration strategies
• Prompt design often requires a few iterations before you get the
desired response consistently.
• Your prompt design strategy should apply the Prompt design best practices,
with incremental refinements.
• You can iteratively introduce some or all of the best practices when testing for
performance that meets your use case needs.
• Additionally, the order of the content in the prompt can sometimes
affect the response.
• Try changing the content order and see how the response changes.
• For example, for multimodal prompts, try adding the files to the prompt
before the instructions.

Ranjitha P-20213CSE0014 46
• As you receive responses from the model, take note of the aspects
that you like and dislike about its responses
• modify your prompts to guide the model to respond that best align
with your use cases.
• Ex: https://
cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/promp
t-iteration

Ranjitha P-20213CSE0014 47
• The natural language instruction in which we interact with an LLM is
called a Prompt
• The inferencing that an LLM does and completes the instruction given
in the prompt is called 'in context learning'
• The ability of the LLM to respond to the instruction in the prompt
without any example is called 'Zero Shot Learning‘
• When a single example is provided, it's called 'One Shot Learning'
• If more than one examples in provided, it's called 'Few Shot Learning‘
• Context Window, or the maximum number of tokens that an LLM can
provide and inference on, is critical in the Zero/One/Few Shot
Learning

Ranjitha P-20213CSE0014 48
Ranjitha P-20213CSE0014 49
Ranjitha P-20213CSE0014 50
Ranjitha P-20213CSE0014 51
Ranjitha P-20213CSE0014 52