AIS421

NLP Applications

Lecture 2: Prompt Engineering

Spring 2024

1
Agenda
Introduction to Prompt Engineering

Basics of Foundation Models

Fundamentals of Prompt Engineering

Prompt Techniques

Prompt and Model Parameters

2
 Introduction to Prompt Engineering

• Imagine you have a powerful tool like a large language model (LLM), but unlocking its full
potential requires knowing how to ask the right questions; That's where prompt engineering
comes in.
• It's the art and science of crafting effective prompts that guide AI models to generate desired
outputs.

3
 Introduction to Prompt Engineering

• The quality and structure of the prompt can significantly influence the foundation model’s
performance on a given task.

4
 Introduction to Prompt Engineering

• At its core, prompt engineering involves:

Understanding AI models: Different models have different strengths and limitations. Knowing
these allows you to tailor your prompts accordingly.
Formulating clear instructions: What exactly do you want the AI to do? Be specific and provide
the necessary context.
Using appropriate language: Employ terms and phrasings the model understands and that align
with your desired outcome.
Considering potential biases: Be aware of potential biases in the training data of the model and
adjust your prompts to mitigate them.

5
 Introduction to Prompt Engineering

• Why is prompt engineering important?

Improved accuracy and relevance: Well-crafted prompts lead to better results, aligned with
your specific needs.
Unlocks diverse outputs: Explore different creative styles, solve problems from various angles,
or generate unique content.
Reduces bias: Mitigate inherent biases in the model by crafting prompts that counterbalance
them.
Expands functionality: Use the model for tasks beyond its initial design by providing precise
instructions.

6
 Introduction to Prompt Engineering

• Prompt engineering shines in different fields:

Creative writing: Generate poems, scripts, or stories in different styles with specific prompts.
Scientific research: Analyze data, write summaries, or propose research questions.
Software engineering: Generate code, debug existing code, or document functionalities.
Marketing and advertising: Craft personalized ad copy, write product descriptions, or generate
social media content tailored to specific audiences.
Education: Create interactive learning materials, answer student questions, or provide
personalized feedback based on specific prompts.

7
 Basics of Foundation Models

• Generative AI is a type of artificial intelligence that can create new content and ideas,
including conversations, stories, images, videos, and music.
• Like all other AI, generative AI is powered by machine learning (ML) models.
• However, generative AI is powered by very large models, commonly called foundation
models (FMs).
• FMs are pretrained on a vast corpus of data, usually through Self-Supervised Learning (SSL)
or Reinforcement Learning from Human Feedback (RLHF).

8
Basics of Foundation Models

9
 Basics of Foundation Models

• Self-supervised learning (SSL):

SSL is a powerful technique in machine learning where a model learns meaningful

representations of data without the need for explicit human-labeled examples.

This makes it particularly useful in situations where labeled data is scarce or expensive
to obtain.

10
 Basics of Foundation Models

• How SSL works in LLMs?:

1-Unlabeled Data as Source:

Instead of relying on labeled data with predefined categories, SSL utilizes large text
corpora devoid of labels.
This opens the door to training on massive datasets like books, articles, and even
social media content.

11
 Basics of Foundation Models

• How SSL works in LLMs?:

2-Crafting Pretext Tasks:

The key lies in designing pretext tasks that encourage the LLM to learn valuable
information from the unlabeled data.
These tasks don't require human-defined labels, as their purpose is to extract
meaningful relationships and patterns from the data itself.

12
 Basics of Foundation Models

• How SSL works in LLMs?:

3-Learning from Self-Generated Signals:

While not directly guided by external labels, the LLM learns by comparing its
predictions against its own internal representations of the data.
This self-supervised approach helps the model develop a stronger understanding of
language structure and semantics.

13
 Basics of Foundation Models

• How SSL works in LLMs?:

Examples of Pretext Tasks:

Masked Language Modeling: Predicting missing words in sentences helps the

LLM understand word relationships and context.
Next Sentence Prediction: Predicting the next sentence in a sequence teaches the
model about coherence and flow.
Contrastive Learning: Comparing similar and dissimilar text pairs helps the model
discern meaningful representations.

14
 Basics of Foundation Models

• Reinforcement Learning from Human Feedback (RLHF):

RLHF is a growing area of research that combines the strengths of reinforcement

learning (RL) with the power of human expertise.

Unlike traditional RL, which relies solely on numerical rewards, RLHF incorporates
human feedback to guide the learning process of AI agents.

15
 Basics of Foundation Models

• How RLHF works in LLMs?:

1- Start with a pre-trained LLM:

The process begins with an LLM that has already been trained on a massive dataset
of text and code.
This gives the LLM a basic understanding of language and the ability to generate
text.

16
 Basics of Foundation Models

• How RLHF works in LLMs?:

2- Define the task:

The task that the LLM will be trained on is clearly defined.

This could be anything from writing different kinds of creative text formats, like
poems or code, to answering questions in a comprehensive and informative way.

17
 Basics of Foundation Models

• How RLHF works in LLMs?:

3- Human feedback loop:

As the LLM generates text, humans provide feedback on its quality, accuracy, and
adherence to the task instructions.
This feedback can be explicit (e.g., ratings or annotations) or implicit (e.g., clicks or
dwell time on generated text).

18
 Basics of Foundation Models

• How RLHF works in LLMs?:

4- Rewarding the LLM:

Based on the human feedback, the LLM receives rewards for generating good
outputs and penalties for generating bad outputs.
This reinforcement signal helps the LLM learn what kind of outputs are desired.

19
 Basics of Foundation Models

• Types of FMs:
Text-to-text models:
Text-to-text models are LLMs that are pretrained to process vast quantities of textual data and
human language.
These large foundation models can summarize text, extract information, respond to questions,
create content (such as blogs or product descriptions), and more.

20
 Basics of Foundation Models

• Types of FMs:
Text-to-image models:
Text-to-image models take natural language input and produce a high-quality image that
matches the input text description.
Some examples of text-to-image models are DALL-E 2 from OpenAI, Imagen from the
Google Research Brain Team, Stable Diffusion from Stability AI, and Midjourney.

21
 Basics of Foundation Models

• New Models:
Sora:
Sora is an AI model that can create realistic and imaginative scenes from text
instructions.
Creates videos up to 60 seconds long based on textual descriptions.

Generates high-quality visuals with intricate details and complex scenes.

 Incorporates diverse camera angles and movements for engaging storytelling.

Features characters with various emotions, allowing for richer narratives.

https://openai.com/sora
22
 Basics of Foundation Models

• New Models:
AnyGPT:

https://arxiv.org/abs/2402.12226
23
 Basics of Foundation Models

• New Models:
NExT-GPT:

https://arxiv.org/abs/2309.05519
24
Fundamentals of Prompt Engineering

25
 Fundamentals of Prompt Engineering

• Prompt engineering is different from fine-tuning.

• In fine-tuning, the weights or parameters are adjusted using training data to optimize a cost
function.
• Fine-tuning can be an expensive process, both in terms of computation time and actual cost.
• Prompt engineering, however, attempts to guide the trained FM, an LLM, or a text-to-image
model, to give more relevant and accurate answers.

26
 Fundamentals of Prompt Engineering

• Elements of a prompt:
Instructions: This is a task for the large language model to do. It provides a task
description or instruction for how the model should perform.
Context: This is external information to guide the model.

Input data: This is the input for which you want a response.
Output indicator: This is the output type or format.

27
 Fundamentals of Prompt Engineering
Prompt Output (Completion)
Write a summary of a service review using two sentences.

Store: Online, Service: Shipping.

Review: Amazon Prime Student is a great option for students looking to save money. Not Amazon Prime Student is a fantastic
paying for shipping is the biggest save in my opinion. As a working mom of three who is also a option for college students, offering
student, it saves me tons of time with free 2-day shipping, and I get things I need quickly and free 2-day shipping, streaming
sometimes as early as the next day, while enjoying all the free streaming services, and books services, books, and other benefits
that a regular prime membership has to offer for half the price. Amazon Prime Student is only for half the price of a regular Prime
available for college students, and it offers so many things to help make college life easier. This membership. It saves time and
is why Amazon Prime is the no-brainer that I use to order my school supplies, my clothes, and money, making college life easier.
even to watch movies in between classes. I think Amazon Prime Student is a great investment
for all college students.

Summary:

28
 Fundamentals of Prompt Engineering

• Instruction: Write a summary of a service review...

• Context: Store: Online, Service: Shipping
• Input data: Review: Amazon Prime Student is a great option for students looking to save money. Not paying for shipping
is the biggest save in my opinion. As a working mom of three who is also a student, it saves me tons of time with free 2-day
shipping, and I get things I need quickly and sometimes as early as the next day, while enjoying all the free streaming services,
and books that a regular prime membership has to offer for half the price. Amazon Prime Student is only available for college
students, and it offers so many things to help make college life easier. This is why Amazon Prime is the no-brainer that I use to
order my school supplies, my clothes, and even to watch movies in between classes. I think Amazon Prime Student is a great
investment for all college students.

• Output indicator: ...using two sentences.

29
 Fundamentals of Prompt Engineering
Best practices for designing effective prompts

• Be clear and concise:

Prompts should be straightforward and avoid ambiguity. Clear prompts lead to more
coherent responses. Craft prompts with natural, flowing language and coherent
sentence structure. Avoid isolated keywords and phrases.

30
 Fundamentals of Prompt Engineering
Best practices for designing effective prompts

• Include context if needed:

Provide any additional context that would help the model respond accurately. For
example, if you ask a model to analyze a business, include information about the type
of business. What does the company do?

31
 Fundamentals of Prompt Engineering
Best practices for designing effective prompts

• Use directives for the appropriate response type:

If you want a particular output form, such as a summary, question, or poem, specify
the response type directly. You can also limit responses by length, format, included
information, excluded information, and more.

32
 Fundamentals of Prompt Engineering
Best practices for designing effective prompts

• Consider the output in the prompt:

Mention the requested output at the end of the prompt to keep the model focused on
appropriate content.

33
 Fundamentals of Prompt Engineering
Best practices for designing effective prompts

• Start prompts with an interrogation:

Phrase your input as a question, beginning with words, such as who, what, where,
when, why, and how.

34
 Fundamentals of Prompt Engineering
Best practices for designing effective prompts

• Provide an example response:

Use the expected output format as an example response in the prompt. Surround it in
brackets to make it clear that it is an example.

35
 Fundamentals of Prompt Engineering
Best practices for designing effective prompts

• Break up complex tasks:

Divide the task into several subtasks. If you cannot get reliable results, try splitting the
task into multiple prompts.
Ask the model if it understood your instruction. Provide clarification based on the
model's response.
If you don’t know how to break the task into subtasks, ask the model to think step by
step.

36
 Fundamentals of Prompt Engineering
Best practices for designing effective prompts

• Experiment and be creative:

Try different prompts to optimize the model's responses.

Determine which prompts achieve effective results and which prompts achieve
inaccurate results.
Adjust your prompts accordingly.
Novel and thought-provoking prompts can lead to innovative outcomes.

37
 Prompt Techniques

• Zero-shot prompting:
A prompting technique where a user presents a task to an LLM without giving the
model further examples.
Here, the user expects the model to perform the task without a prior understanding, or
shot, of the task. Modern LLMs demonstrate remarkable zero-shot performance.
The larger the LLM, the more likely the zero-shot prompt will yield effective results.

Instruction tuning can improve zero-shot learning. You can adopt reinforcement
learning from human feedback (RLHF) to scale instruction tuning.

38
Prompt Techniques

39
 Prompt Techniques

• Few-shot prompting:
A prompting technique where you give the model contextual information about the
requested tasks.
In this technique, you provide examples of both the task and the output you want.

Providing this context, or a few shots, in the prompt conditions the model to follow the
task guidance closely.
The label distribution of the input text specified by the demonstrations is important.

40
Prompt Techniques

41
 Prompt Techniques

• Chain-of-thought prompting:
Chain-of-thought (CoT) prompting breaks down complex reasoning tasks through
intermediary reasoning steps.
You can use both zero-shot and few-shot prompting techniques with CoT prompts.

Chain-of-thought prompts are specific to a problem type.

You can use the phrase "Think step by step" to invoke CoT reasoning from your
machine learning model.

42
Prompt Techniques

43
Prompt Techniques

44
 Prompt Techniques

• Self-consistency:
A prompting technique that is similar to chain-of-thought prompting.

However, instead of taking the obvious step-by-step, or greedy path, self-consistency

prompts the model to sample a variety of reasoning paths.
Then, the model aggregates the final answer based on multiple data points from the
various paths.

45
Prompt Techniques

46
 Prompt Techniques

47
https://arxiv.org/abs/2203.11171
 Prompt Techniques

• Tree of thoughts:
Another technique that builds on the CoT prompting technique.

 CoT prompting samples thoughts sequentially, but ToT prompting follows a tree-
branching technique.
With the ToT technique, the LLM can learn in a nuanced way, considering multiple
paths instead of one sequential path.

48
 Prompt Techniques

• Tree of thoughts:

49
https://arxiv.org/abs/2305.10601
 Prompt Techniques

• Tree of thoughts:

50
https://arxiv.org/abs/2305.10601
 Prompt Techniques

• Tree of thoughts:
For Game of 24, Generative Pre-trained Transformer 4 (GPT-4) achieved a 4 percent
success with CoT prompting. However, the model reached 74 percent success with a
ToT prompting method.

51
https://arxiv.org/abs/2305.10601
 Prompt Techniques

• Tree of thoughts:

52
https://arxiv.org/abs/2305.10601
 Prompt Techniques

• Advanced techniques such as RAG, ART, and ReAct will be covered in a separate lecture.

53
 Prompt and Model Parameters

• Prompt Length: The length of the prompt, including the number of tokens or characters, can
impact the context provided to the model and influence the generated output.

• Prompt Format: The format of the prompt, such as the structure of the input text, the
inclusion of keywords or instructions, and the use of special tokens, can guide the model in
generating relevant responses.

54
 Prompt and Model Parameters

• Temperature: Temperature controls the randomness of the generated output. Lower

temperatures result in more deterministic and conservative responses, while higher
temperatures lead to more diverse and creative responses.
• High temperature (T > 1): More creative and diverse outputs, but potentially less factual and
coherent. Can be useful for generating imaginative text formats, jokes, or poems.
• Low temperature (T < 1): More conservative and predictable outputs, closer to the LLM's
learned patterns. Suitable for factual summaries, instructions, or code generation.

55
 Prompt and Model Parameters

• Top_p: adjusts determinism with "nucleus sampling." Lower values give exact answers, while
higher values give diverse responses. This value controls the diversity of the model's
responses.

• Top_k: is the number of the highest-probability vocabulary tokens to keep for top- k-filtering.
Similar to the Top_p parameter, Top_k defines the cutoff where the model no longer selects the
words.

56
 Prompt and Model Parameters

• Repetition Penalty: Repetition penalty discourages the model from repeating tokens it has
already generated in the output sequence, which can help produce more diverse and coherent
responses.

• Length Penalty: Length penalty encourages the model to produce longer or shorter output
sequences by adjusting the likelihood of longer or shorter sequences during generation.

57
 Prompt and Model Parameters

• Special Tokens: Special tokens, such as <BOS> (beginning of sequence) and <EOS> (end of
sequence), can be used to mark the start and end of prompts, instructions, or sections within
the input text.
• Context Window Size: The size of the context window, or the number of previous tokens
considered by the model during generation, can impact the coherence and relevance of the
generated output.
• Fine-Tuning Parameters: Parameters specific to fine-tuning, such as learning rate, batch size,
and number of training epochs, can affect how well the model adapts to specific prompts and
tasks.

58
 Best Courses

• The primary content covered in this lecture draws heavily from the curriculum of the course
titled "Foundations of Prompt Engineering," accessible at the following link:
https://explore.skillbuilder.aws/learn/course/internal/view/elearning/17763/foundations-of-prom
pt-engineering

• I would like to suggest incorporating the course "ChatGPT Prompt Engineering for
Developers" into your considerations. You can find more information about this course at the
following link:
https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/

59
 AIS421

NLP Applications

Lecture 2: Prompt Engineering

Spring 2024

60