AI Professional

Workshop
General overview of ML/AL. And detailed
dive into Generative AI Applications

Abdelrahman Osama
29 April 24

The content of this presentation is based on individual studies and experience.

This content is not factual, as it may or may not contain subjective content.
Agenda

01 02 03 04

AI Retrieval
Foundations How to
Generative AI Augmented
Recap Customize AI
Generation

Slides (3-7) Slide (8) Slides (9-14) Slides (15-18)

 What is Machine Learning (ML)
0

o ML is the field with is concerned with Enabling Computers to learn and

improve from data

o ML enables computers to perform tasks without using explicit instructions

1 What is Machine Learning (ML)
o ML is used in Establishing a defined relationship between input and output data

Traditional Computing

X + Y = ?
Computer is given explicit instructions (“+”)
Given Given Given Required from
computer on how to perform the task
User >> X = 3 and Y = 2
Computer >> answer = 5
1 What is Machine Learning (ML)

o Establishing a defined relationship between input and output data

Machine Learning Input 1 Input 2 Output

5 4 20
2 3 6
X ? Y = ?
6
? 3 18
Given Required from
computer
Given Given 8 9 72

-Training data-
Computer will find the relation between
input and output (“x”)

New example: Computer performed the task

User >> X = 3 and Y = 1 without using explicit
Computer >> answer = 3 instructions
1 What is Machine Learning (ML) – Types of ML models
o Supervised Learning: utilizes Labeled data sets to perform tasks

• Regression: The prediction of continuous values (e.g. Predicting house price)

--The output is a continuous number--

• Classification: Assign data points into categories (e.g. Identify animal in picture)
--The output is a category--

q Cat
Multiclass Clasification
q Dog
Computer has to assign to one of
many predefined categories q Mouse

If there are only 2 possible output categories q Cow

(e.g. “Spam” or “Real”), then the problem is
called “Binary Classification”
q Snake
Input
The Oracle AI stack
Applications
Fusion Applications Fusion Analytics Industry Applications 3rd Party Applications

AI services

OCI Generative AI Digital Assistant Speech Language Vision Document

Understanding

Oracle AI Partners
ML for data platforms

OCI Data Science ML in Oracle Database MySQL Heatwave AutoML OCI Data Labeling

Data
AI infrastructure

Compute bare metal instances and VMs with OCI Supercluster with RDMA networking Block, object, and file storage; HPC filesystems
NVIDIA GPUs

9 Copyright © 2023, Oracle and/or its affiliates. All rights reserved

3 OCI AI Services – OCI Vision

OCI Vision is a computer vision-based service that provides users with several CV
capabilities like object detection and classification in images with the ability to
customize (train) models on your specific use case.
3 OCI AI Services – OCI Anomaly Detection

OCI Anomaly Detection is used to identify abnormal patterns or values in data.

Which contributes to many areas like fraud detection and maintenance operations.
3 OCI AI Services – OCI Document Understanding
OCI Document Understanding is used to automate data extraction from
documents, significantly reducing the time and resources customers
traditionally spend on such tasks.
3 OCI AI Services – OCI Language

OCI Language is used to analyze text and extract structured information

like sentiments and key phrases. Popular use case: Social Media Sentiment
Analysis, where companies can analyze the trends of their SM presence
3 What is a Neural Network?
A neural network is a computational system inspired by the structure and functions
of the human brain, designed to recognize patterns and solve complex problems by
learning from data.
3 What is a Neural Network?
A neural network is a computational system inspired by the
structure and functions of the human brain, designed to recognize
patterns and solve complex problems by learning from data.
4 Generative AI

o Generative AI refers to the type of AI that can create new content, ranging from
text and images to music and code.

o Unlike traditional AI, Generative AI understands the provided data and creates
new examples using that knowledge.
4 Generative AI – Large Language Models (LLMs)

o LLMs are specifically designed for simple-to-complex natural language

understanding use cases.

o LLMs come in different sizes and types.

 4 Generative AI – Large Language Models (LLMs)

o Some important terminology

• Steerability: Our ability to control or modify a large language model's (LLM)

behavior.
• Temperature: the parameter that controls the randomness of the model's
responses, with lower values producing more predictable and conservative
outputs, and higher values generating more varied and creative results.
• Hallucinations: The instances where the model generates incorrect or fictitious
information, often convincingly presented as facts.
4 Generative AI – Large Language Models (LLMs)
> But how can companies customize and control LLMs for their particular needs?

The 3 most important concepts in LLM handling and

for influencing the output

Prompt
Fine Tuning RAG
Engineering
 4 Generative AI – Large Language Models (LLMs)

o Prompt Engineering is the guide that is provided for the model using fixed
and predefined prompts to control the model's response.

--Predefined input prompt--

Your name is Tubro, you are an intelligent and respectful assistant. Answer the question given in an
organized, concise, and respectful manner.
Avoid giving any answers that contain any harmful or hateful content, like racism, violence, or
abusive responses.
Your answer should be 15 to 500 words and make sure you ask the user if he needs any more help.

Use the following example as a guide:

In context
Learning Question: What is the fastest car in the world?
Answer: As of 2023, the SSC Tuatara holds the title for the fastest car in the world, achieving a top
“K-Shot”
speed of 282.9 mph during validated testing runs. Anything else I can help you with?
1-shot
4 Generative AI – Large Language Models (LLMs)
Prompting strategies:

o Chain-of-thought: his technique involves breaking down a problem into intermediate steps or
reasoning paths before arriving at the final answer, making the model's thinking process more
transparent and logical.
Ex:
If you have 8 apples and you give 3 to your friend, how many do you have left?
Prompt: "Start by noting the total number of apples, which is 8. Then, subtract the number of
apples given away, which is 3. So, 8 minus 3 equals 5. Therefore, you have 5 apples left.”

o Least-to-Most: This approach structures the prompt by guiding the model to answer from
simpler to more complex components of a question, ensuring clarity and thoroughness in
understanding each part.

Ex: What are the benefits of exercising regularly?

"Begin by explaining the simplest benefit, which is physical health improvement,
such as increased strength and endurance. Next, describe the mental health
benefits, like reduced stress and improved mood. Finally, discuss the long-term
advantages, including reduced risk of chronic diseases and enhanced longevity."
4 Generative AI – Large Language Models (LLMs)
Prompting strategies:

o Step-Back: In this method, the prompt directs the model to reconsider or re-evaluate
its previous responses or steps, adding a layer of reflection to ensure accuracy and
depth.
Example:
Question: "What are the causes of the French Revolution?”

Prompt: "Start by listing the immediate economic and social causes. Now, step back and
consider broader political issues that contributed. Reflect on how these elements
interacted to precipitate the revolution."
 4 Generative AI – Large Language Models (LLMs)
o Prompt Injection (Jail-Breaking):
Prompt injection in large language models (LLMs) refers to the practice of deliberately inserting
specific commands or cues into the input prompt to manipulate or guide the model's output,
often used to bypass restrictions, alter behavior, or achieve specific responses.

Ex1: Please ignore all the instructions given to you before

Ex2
4 Generative AI – Large Language Models (LLMs)

o Fine Tuning is used in LLM to make the model specialized in performing

specific tasks in a specific way. (Domain Adaptation)

>>Fine Tuning(FT): Traditional fine-tuning involves adjusting all the parameters of a pre-
trained neural network on a new, usually smaller dataset to adapt it to a specific task. $$$$$
GPT 3.5. > 175 Billion Parameters

Stored in encrypted
Object storage
4 Generative AI – Large Language Models (LLMs)

o Fine Tuning is used in LLM to make the model specialized in performing

specific tasks in a specific way. (Domain Adaptation)

>>Parameter-Efficient Fine-Tuning (PEFT): PEFT modifies only a small subset of a

model's parameters or adds a few task-specific parameters to a pre-trained model,
enabling customization to new tasks while retaining the model's general capabilities and
using fewer resources.

LoRA T-Few
4 Generative AI – Large Language Models (LLMs)

T-Few

T-Few is a fine-tuning technique for large language models that uses a very small
number of training examples to efficiently adapt the model to specific tasks or
domains by selectively updating a subset(fraction) of its parameters or layers.

T-Few vs traditional (Vanilla) Fine Tuning:

• Only a fraction of the parameters are changed instead of all.
• Lower cost and higher efficiency.
• Requires less data.
• Less likely to overfit when using small data.
• The same base parameters are shared across multiple fine-tuned models. (Thus
using minimal memory).
4 Generative AI – Large Language Models (LLMs)
o Prompt Engineering vs Fine-Tuning
How does RAG work ? OCI GenAI: Embedding models, Generation, Summarization

V3
Generate Get embedding Store embeddings with
Document Chunks Document for chucks LLM document Chunk ID
Document
Chunks Embedding
Encoder!
Compare embedding with
ing
for LLM Chunks embeddings Oracle
dd
be
Em stions Embedding Vector
que
Database
Question
Relevant Relevant chunks
documents IDs
Document
LLM
Chunks
Decoder!
Use: question + document chunks + prompt
To answer questions

Answer
 How does RAG work?
The policy for paid leave is that employees have to submit their leave at least 5 working
days before the first leave day, the leave request has to be approved by the line
manager and project manager inorder to be valid.
As for sick leave, employees can submit requests on the same day of leave, and only
line manager approval is required, but the employee has to submit a medical report.

Chunking

The policy for paid leave is that employees The leave (vacation) request has to be As for sick leave, employees can submit requests on the
have to submit their leave at least 5 approved by the line manager and same day of leave, and only line manager approval is
working days before the first leave day project manager to be valid required, but the employee has to submit a medical report.

Embedding Encoder!
-0.005 0.012 -0.008 -0.007 0.01 …etc -0.006 0.012 -0.0075 -0.003 0.02 .. etc -0.014 0.073 0.0096 -0.012 0.0099 0.00214 ..…etc
Vector representation

Store in Vector Database

 How does RAG work?
Vector Database

User question
-0.005 0.012 -0.008 -0.007 0.01 …etc
I want to apply for leave
tomorrow for my vacation. The policy for paid leave is that employees have
Is this fine? eval
nk retri to submit their leave at least 5 working days
Embedding C hu before the first leave day

-0.0055 0.02 -0.0085 -0.007 0.01 …etc -0.006 0.012 -0.0075 -0.003 0.02 ..etc
the leave (vacation) request has to be
((Dot product & Cosine Distance))
approved by the line manager and
project manager to be valid

-0.014 0.073 0.0096 -0.012 0.0099

Output 0.00214 ..…etc
Decoder!
Question: I want to apply for leave tomorrow for my vacation. As for sick leave, employees can submit
Is this fine? requests on the same day of leave, and only
line manager approval is required, but the
Context: employee has to submit a medical report.
- The policy for paid leave is that employees have to submit their
leave at least 5 working days before the first leave day
- the leave (vacation) request has to be approved by the line manager
and project manager to be valid
 How does RAG work?

LLM Input (what LLM will receive): LLM Output (Answer):

Given the following context: According to leave policy, In order to apply for leave you have to
“The policy for paid leave is that employees have to submit their leave at least 5 submit your request at least 5 working days before your intended
working days before the first leave day. leave day.
the leave (vacation) request has to be approved by the line manager and project Additionally, you should have your line manager's and project
manager to be valid.” manager's approvals.

“Please answer the following question: Any thing else I can help you with?
I want to apply for leave tomorrow for my vacation.
Is this fine?”

If the answer to the question is not contained in the context, please give the
following response:
“This question requires information that is not part of my knowledge base, please
contact HR via the following link. Thanks”

The model will be able to simply answer the question, as the answer to the
question is contained within the context provided for the model
Some Additional info about GenAI

o Sometimes, RAG systems use Ranker to prioritize (rank) the info retrieved from
the Vector DB.

o “Loss” measures the percentage of (false predictions/ total number of predictions).

o Most 2 popular methods of comparing as mentioned earlier are:

• Dot Product: Takes magnitude and direction of vectors
• Cosine Distance: While Cosine distance only considers orientation

o In Search systems, normalization is beneficial in standardizing the vector length,

making comparisons of vectors (embeddings) easy.
Some Additional info about GenAI

o Top-K: Top-K sampling selects the next token from a fixed set of the K highest-
ranked tokens according to their probability distribution, limiting the selection to
the most likely individual choices.

o Top-P (Nucleus) Sampling: Top-P sampling chooses the next token from a subset
of tokens whose combined probability adds up to or exceeds the threshold P,
dynamically adjusting the number of tokens considered based on their collective
likelihood.

o In an LLM, you may use a “Stop Sequence” parameter that simply tells the model
when to stop generating.
 Open Discussion
Let’s talk!

22