Whereas retrieval-augmented generation (RAG) gives you one way to provide additional

information to a large language model (LLM), there’s another technique called fine-tuning,
which offers another approach to enrich the model. Fine-tuning is particularly useful when the
context exceeds the input window length of the LLM. By fine-tuning, you can help an LLM
absorb additional information and even tailor its output to a specific style. However,
implementing fine-tuning is generally more complex than RAG. Let’s explore how this
works.

Imagine you have an LLM trained on data from the internet, such as the sentence "My
favorite food is a bagel with cream cheese." This LLM has likely learned from hundreds of
billions—or even trillions—of words to predict the next word in a sequence. Such models are
pre-trained on massive datasets, a process commonly called pre-training. Now, suppose we
want the LLM to exhibit a relentlessly positive and optimistic attitude. To achieve this, we
can use fine-tuning to modify its behavior.

Fine-tuning involves creating a dataset with sentences that embody the desired style or
attitude. For instance, you could compile examples like "What a wonderful chocolate cake!"
or "The novel was thrilling." These sentences allow the LLM to predict subsequent words in a
context consistent with positivity and optimism. Even with a relatively modest additional
dataset of 10,000 to 1,000,000 words, fine-tuning can significantly shift the LLM's outputs
towards a positive, optimistic tone.

While having an LLM with an optimistic outlook might seem trivial, fine-tuning has practical
applications. One notable use case is when a task is challenging to define through prompts
alone. For example, if you want an LLM to summarize customer service calls, a generic
output might summarize a call as "The customer mentioned an issue with a monitor."
However, for a call center, you may need a more detailed and specific summary, like "The
MK-4127 KX monitor was reported broken by customer 542." By fine-tuning the LLM with
hundreds of carefully written examples, you can train it to produce summaries in the desired
style.

Fine-tuning is also effective for tasks like mimicking a specific writing or speaking style. For
instance, creating an LLM that sounds like a particular individual can be difficult to achieve
with prompts because personal styles are hard to articulate clearly. A better approach is to
fine-tune the LLM on transcripts of that person’s actual speech or writing. For example, if you
wanted the LLM to emulate my speaking style, you could fine-tune it using my transcripts.
This method allows the model to generate outputs that closely resemble my way of speaking,
which is otherwise challenging to define with prompts alone.

In conclusion, fine-tuning provides a powerful, precise method for adapting LLMs to specific
tasks, styles, or tones, especially when such adaptations are not easily described in a prompt.
Whether for summarization, stylistic mimicry, or domain-specific tasks, fine-tuning offers a
valuable tool for customizing LLM capabilities.

Describing a specific person's style through written text instructions can be challenging. Fine-
tuning, however, proves to be a more effective way to get a language model to adopt and
consistently use a certain style. For instance, if you're building an artificial character—
perhaps a cartoon character—fine-tuning can enable the model to speak in a specific,
consistent style that aligns with the character's persona.

Beyond tasks that are difficult to define in prompts, a second broad application of fine-tuning
involves enabling a language model to gain expertise in a specific domain. For example, if
you want a language model to process medical notes, it needs to handle domain-specific
terminology and structures. A typical medical note might include shorthand and technical
language, such as:

 "PT is patient."
 "CO: complaining of SOB (shortness of breath), DOE (dyspnea on exertion)."
 "PE: physical examination findings."

This kind of language is far from standard English, and a language model trained on general
English text would struggle to understand and process it effectively. Fine-tuning the model
with a dataset of medical records enables it to absorb this specialized knowledge, allowing for
the development of applications like medical record analysis or summarization.

Similarly, fine-tuning can be applied to other specialized domains, such as legal documents.
Legal texts often include formal and arcane language, like "hereto," "thereof," or "non-
exclusive rights under section 2(a)(3)." This style of writing is typically inaccessible to those
without legal training. By fine-tuning on a corpus of legal documents, the language model can
learn to interpret and process such texts, improving its performance in legal applications.

Financial documents, another challenging domain, can also benefit from fine-tuning. These
documents are filled with specific terminology, numerical data, and structured layouts. A fine-
tuned language model could enhance its understanding and handling of financial data, making
it more effective for tasks like generating reports or analyzing trends.

Another important reason to fine-tune is to enable smaller models to perform tasks that
traditionally require larger models. While large models, often with over 100 billion
parameters, can handle complex reasoning and large knowledge bases, they come with
downsides such as higher latency, greater computational requirements, and increased costs.
Running such models often requires specialized hardware like GPU servers, making them
impractical for deployment on everyday devices like laptops or smartphones.

By fine-tuning a smaller model, say one with 1 billion parameters, on a specific dataset, it can
be optimized to perform well on simpler tasks, like classifying restaurant reviews as positive
or negative. This approach reduces costs and makes the model more accessible for everyday
use. While smaller models aren't as inherently capable as larger ones, fine-tuning allows them
to perform specific tasks at a high level of accuracy, provided there is enough domain-specific
data available.

To summarize, fine-tuning offers a versatile technique for improving the capabilities of a

1. Tasks that are hard to define in prompts, such as mimicking a specific writing style.
2. Gaining specialized knowledge in domains like medicine, law, or finance.
3. Enabling smaller and cheaper models to perform tasks efficiently, reducing the need
for large models.

Fine-tuning, along with techniques like retrieval-augmented generation (RAG), provides a