Beginner’s Guide to RAG

AI by Hand ✍

Prof. Tom Yeh

Hosted by:
 Download Slides

https://by-hand.ai/rag

2
Roadmap

Q/A
RAG Equation
Load
Prompt Advanced
Split
Index Retrieval
Embed

Store
RAG Equation
Beginner’s Guide to RAG - AI by Hand ✍

Download: https://by-hand.ai/rag
 RAG = Query + Prompt + Context + LLM
How much does [INST] <<SYS>> You are a _______, respectful, ______
Tom make? assistant. Always answer as helpfully as possible,
while being safe. Your answers should ___ include
Search any harmful, _____________, racist, sexist, toxic,
dangerous, or illegal content. Please ensure that
your responses are socially _________ and positive
in nature. <</SYS>>

Please answer the following question by using

Database information from the provided context information!
CONTEXT_INFORMATION:
LLM
Amy, 100, CEO
Tom, 50, Prof
Sam, 120, CTO QUESTION:

[/INST]

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 RAG = Query + Prompt + Context + LLM
How much does [INST]
Tom make? Please answer the following question by using
information from the provided context information!
Search CONTEXT_INFORMATION:

QUESTION:

[/INST]
Database LLM
Amy, 100, CEO
Tom, 50, Prof
Sam, 120, CTO

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 RAG = Query + Prompt + Context + LLM
How much does [INST]
Tom make? QUESTION:

Search [/INST]

LLM

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

Prompt
Beginner’s Guide to RAG - AI by Hand ✍

Download: https://by-hand.ai/rag
 Summarize
Context information from ________ sources is below.
---------------------

{context_str}

---------------------

Given the information from multiple sources and not

__________________, answer the query.

Query: {query_str}

Answer:

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 Single Choice
Some choices are given below. It is provided in a numbered list
(1 to {num_chunks}),
where each item in the list corresponds to a summary.
---------------------

{context_list}

---------------------
Using only the choices above and not prior knowledge, return the
_________ that is most relevant to the question:

{query_str}

Provide choice in the following format: 'ANSWER: <number>' and

____________ why this choice was selected in relation to the
question.

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 Refine
The original query is as follows: {query_str}
We have provided an existing answer: {existing_answer}
We have the opportunity to _________ the existing answer
with some more context below.
------------

{context_str}

------------
Given the new context, refine the original answer to better answer
the query. If the context isn't useful, return the _________ answer.

Refined Answer:

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

Index
Beginner’s Guide to RAG - AI by Hand ✍

Download: https://by-hand.ai/rag
 id: 1 id: 2 id: 2

Load
date: 5/10 date: 5/15 date: 5/17

大人吃大包 每天都吃一 高山，大海

子，小孩吃 個蘋果，可 ，森林，動
小包子 以保持健康 物，大自然

Split 每天都
吃一個
蘋果可
高山大
大海森
林動物
以保持 大自然

Embed 1 1 0 1 1 -2
1 0 1 1 1 1
0 0 1 0 0 1
0 1 0 0 1 1

Store Chunk Embed Meta

{id:_, date: _____}
{id:_, date: _____}

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

Load
Beginner’s Guide to RAG - AI by Hand ✍

Download: https://by-hand.ai/rag
 LlamaHub Data Loaders

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 Which loader to use to read CSV data?

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 Which loader to use to read Word documents?

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

Split
Beginner’s Guide to RAG - AI by Hand ✍

Download: https://by-hand.ai/rag
 Split by Tokens
How are you doing today? I hope you're doing
TokenTextSplitter( well. Whether your day is busy or calm, I hope
chunk_size=4, you find a moment of joy. Life moves fast, with
chunk_overlap=0 many ups and downs. It's important to stop and
) enjoy the little things. Maybe today brought a
good chat, a tasty meal, or a beautiful sunset.
Cherish these happy moments. Each day is a
chance to make good memories and connect
with others. Take time to appreciate your
surroundings and the people in your life. Even
on challenging days, there are opportunities to
learn and grow. Let's embrace each experience
and look forward to tomorrow with optimism.

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 Split by Tokens
How are you doing today? I hope you're doing
TokenTextSplitter( well. Whether your day is busy or calm, I hope
chunk_size=4, you find a moment of joy. Life moves fast, with
chunk_overlap=1 many ups and downs. It's important to stop and
) enjoy the little things. Maybe today brought a
good chat, a tasty meal, or a beautiful sunset.
Cherish these happy moments. Each day is a
chance to make good memories and connect
with others. Take time to appreciate your
surroundings and the people in your life. Even
on challenging days, there are opportunities to
learn and grow. Let's embrace each experience
and look forward to tomorrow with optimism.

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

Embed
Beginner’s Guide to RAG - AI by Hand ✍

Download: https://by-hand.ai/rag
 Massive Text Embedding Benchmark
https://huggingface.co/spaces/mteb/leaderboard

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 Which models have 7B parameters?

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 Which model has the most parameters?

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 Which model supports the longest context
window?

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 Which model has the best classification
performance?

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 Which model has the best clustering
performance?

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag
Store
Beginner’s Guide to RAG - AI by Hand ✍

Download: https://by-hand.ai/rag
 Choose a Vector Database
https://superlinked.com/vector-db-comparison

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 Which database can fit 3072 dimensions?

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

Retrieval
Beginner’s Guide to RAG - AI by Hand ✍

Download: https://by-hand.ai/rag
 SQL – Retrieve by Similarity

________ comment, emb<___>[0, 2, 1, 1] AS score

FROM posts
ORDER BY ______ ASC | DESC ;

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 High-level API: Superlinked

query = Query(post_index)
._________(post)
._________(relevance_space.text, Param("________"))

app.query(query, _________ = "how are you?" )

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

Advanced RAGs
Beginner’s Guide to RAG - AI by Hand ✍

Download: https://by-hand.ai/rag
 RAG
DB
Prompt

doc-
doc-
doc-

Query

LLM
AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag
 Multiple Embedding Spaces
DB
Prompt

doc-
doc-
doc-

Query

LLM
AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag
 Superlinked Vector Compute
Memory

Search

ID Description Headline Recency

Output 1
2
0 1 1 0 0 1 0 1 0

1 1 0 0 1 0 0 1 0

3
df = result._________() 0 1 1 1 0 0 1 0 1

__________(df)

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 RankGPT
DB
Prompt

doc-21
doc-42
doc-39

Query

LLM
AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag
 RankGPT
Search Query: {query}.

Rank the {num} passages above based on their _____________ to the

search query. The passages should be listed in _____________ order
using identifiers.

The most relevant passages should be listed first.

The output format should be [] > [], e.g., [1] > [2].

Only response the ranking results,

do not ___________.

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 Multi-Query Retrieval
DB
Prompt

doc-21
doc-42
doc-39

Query

LLM
AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag
 Multi-Query Retrieval
You are an AI language model assistant. Your task is to generate ____
different versions of the given user question to retrieve relevant
documents from a vector database.

By generating multiple _____________ on the user question, your goal

is to help the user overcome some of the limitations of _____________
search.

Provide these alternative questions seperated by newlines.

Original question: {question}

AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag

 Contextual Compression
DB
Prompt

doc-21
doc-42
doc-39

Query

LLM
AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag
 Hypothetical Document Embeddings (HyDE)
DB
Prompt

doc-
doc-
doc-

Query

LLM
AI by Hand ✍ 2024 © Tom Yeh Download: https://by-hand.ai/rag
Q/A
Beginner’s Guide to RAG - AI by Hand ✍

Download: https://by-hand.ai/rag