Scribd
Upload
228 views9 pages

Guide To Top 7 LLM Parameters

GENAI

Uploaded by

vicky.sonawane3
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
228 views9 pages

Guide To Top 7 LLM Parameters

GENAI

Uploaded by

vicky.sonawane3
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 9

Guide to

Top 7
LLM Generation
Parameters

Dipanjan (DJ)
Max Tokens

The max_tokens parameter controls the length of the output


generated by the model
A “token” can be as short as one character or as long as one word
By setting an appropriate max_tokens value, you can control
whether the response is a quick snippet or an in-depth explanation
Max_token value is now deprecated in favor of
max_completion_tokens (in OpenAI API)

Source: 7 LLM Parameters to Enhance Model Performance - Analytics Vidhya


Temperature

The temperature parameter influences how deterministic or


random and creative the model’s responses are
It’s essentially a measure of how deterministic the responses
should be:
Low Temperature (e.g., 0.1): The model will produce more focused and predictable
responses.
High Temperature (e.g., 0.9): The model will produce more creative, varied, or even
“wild” responses.
Use low temperatures for tasks like generating technical answers,
where precision matters, and higher temperature for creative
content generation tasks
Source: 7 LLM Parameters to Enhance Model Performance - Analytics Vidhya
Top-p - Nucleus Sampling

The top_p parameter, also known as nucleus sampling, helps


control the diversity of responses
It sets a threshold for the cumulative probability distribution of
next token generation choices:
Low Value (e.g., 0.1): The model will only consider the top 10% of possible next
tokens, limiting variation.
High Value (e.g., 0.9): The model considers a wider range of possible next tokens
(summing up to 90%), increasing variability

Source: 7 LLM Parameters to Enhance Model Performance - Analytics Vidhya


Top-k - Token Sampling

The top_k parameter limits the model to only considering the top k
most probable next tokens when predicting (generating) the next
word
Low Value (e.g., 10): Limits the model to more predictable and constrained
responses
High Value (e.g., 100): Allows the model to consider a larger number of tokens,
increasing the variety of responses
The top_k parameter isn’t directly available in the OpenAI API but
is available in other platforms like Hugging Face transformers

Source: 7 LLM Parameters to Enhance Model Performance - Analytics Vidhya


Frequency Penalty

The frequency_penalty parameter discourages the model from


repeating previously used words. It reduces the probability of
tokens that have already appeared in the output
Low Value (e.g., 0.0): The model won’t penalize for repetition
High Value (e.g., 2.0): The model will heavily penalize repeated words,
encouraging the generation of new content
This is useful when you want the model to avoid repetitive outputs,
like in creative writing, where redundancy might diminish quality

Source: 7 LLM Parameters to Enhance Model Performance - Analytics Vidhya


Presence Penalty

The presence_penalty parameter is similar to the frequency


penalty, but instead of penalizing based on how often a word is
used, it penalizes based on whether a word has appeared at all in
the response so far
Low Value (e.g., 0.0): The model won’t penalize for reusing words
High Value (e.g., 2.0): The model will avoid using any word that has already
appeared
Presence penalty helps encourage more diverse content generation

Source: 7 LLM Parameters to Enhance Model Performance - Analytics Vidhya


Stop Sequence

The stop parameter lets you define a sequence of characters or


words that will signal the model to stop generating further content
This allows you to cleanly end the generation at a specific point.
Example Stop Sequences: Could be periods (.), newlines (\n), or specific phrases
like “The end”.
Useful especially if you teach the model to generate content until a
specific special token when fine-tuning

Source: 7 LLM Parameters to Enhance Model Performance - Analytics Vidhya


Hands-on Guide

Check out the


HANDS-ON GUIDE
here

You might also like