Unit_5

***** : Please refer the Notes given to you by L&T

Chain-of-thought (CoT) with LLM:

Chain-of-thought (CoT) reasoning allows LLMs to break down complex problems into
intermediate reasoning steps, improving accuracy and logical consistency.

Benefits of CoT Reasoning:

 Encourages step-by-step breakdowns for complex queries.

 Reduces hallucinations by verifying each step.
 Mimics human problem-solving methods.

Example: Word Problem Solving

Question: "If John has 3 apples and buys 5 more, then gives away 2, how many does he have
left?"

Without CoT:

 "John has 6 apples" (Incorrect).

With CoT:

1. John starts with 3 apples.

2. He buys 5 more: 3 + 5 = 8.
3. He gives away 2: 8 - 2 = 6.
4. Final answer: 6 apples.

This structured approach improves accuracy and reasoning.

ReAct (Reasoning + Action)

ReAct (Reasoning + Action) is a framework that enables large language models (LLMs) to
reason step-by-step while also performing actions, such as interacting with APIs or external
applications. Unlike traditional LLM approaches, which either generate direct answers or rely
on external tools passively, ReAct allows models to dynamically integrate reasoning with
action execution.

How ReAct Works:

1. Reasoning: The LLM thinks through the problem, breaking it into logical steps.
2. Action: Based on reasoning, the LLM interacts with external applications (e.g., search
engines, databases, APIs).
3. Observation: The LLM processes the external feedback and updates its reasoning.
4. Iteration: Steps 1-3 repeat until a final answer is reached.

Example: LLM Using ReAct for Weather Prediction

Scenario: A user asks, "What will the temperature be in New York tomorrow?"

 Step 1 (Reasoning): The LLM determines that it needs real-time weather data.
 Step 2 (Action): The LLM calls a weather API: fetch_weather("New York",
"tomorrow").
 Step 3 (Observation): The API returns {"temperature": 22°C, "condition":
"Sunny"}.
 Step 4 (Final Response): The LLM responds, "The expected temperature in New York
tomorrow is 22°C with sunny weather."

Thus, ReAct enables LLMs to enhance decision-making by dynamically combining

reasoning and action execution.

RAG: (Applications: LLMs)

Retrieval Augmented Generation (RAG) enhances the performance of LLM-powered

applications by combining external knowledge retrieval with language generation. This
approach helps improve response accuracy, reduces hallucinations, and ensures that the
model can provide up-to-date and contextually relevant information.
How RAG Works in LLM Applications:
1. User Query Processing: The user provides an input prompt.
2. Retrieval Step: The system searches relevant external data sources, such as databases,
documents, or web pages, to fetch useful information.
3. Augmenting the LLM: The retrieved data is combined with the original prompt and
passed to the LLM.
4. Response Generation: The LLM generates a response based on both the retrieved
information and its pre-trained knowledge.
Example:
Scenario: A legal research chatbot using an LLM.
 Without RAG: The chatbot might generate legal explanations based only on its
training data, which may be outdated.
  With RAG: The chatbot retrieves recent court rulings and updated laws from a legal
database, integrates them with the user's query, and generates an accurate response
based on current legal standards.
Thus, RAG significantly enhances the reliability and factual accuracy of LLM applications
by grounding responses in external, verified sources.

LLM integration into real-world applications

To integrate an LLM into a chatbot or summarization tool (for example), the following steps
are taken:

1. Define the Use Case: Determine whether the LLM will be used for customer support,
text summarization, etc.
2. Choose an LLM: Select a suitable model (e.g., GPT for chatbots, BART for
summarization).
3. Develop a Prompting Strategy: Fine-tune the model with domain-specific prompts.
4. Optimize Performance: Implement caching, rate limiting, and token optimization to
enhance response time.
5. Integrate External APIs: Use real-time data sources for better responses.
6. Monitor & Update: Continuously improve based on user feedback.

By considering these factors, LLMs can be effectively deployed in real-world applications.

These questions and answers provide a structured understanding of LLM-powered
applications, architectures, reasoning techniques, and practical implementations.

Program-Aided Language Models (PAL)

Program-Aided Language Models (PAL) enhance reasoning by integrating traditional

language models with code execution capabilities. Instead of just predicting text, PAL allows
LLMs to generate and run programs (e.g., Python code) to compute answers more accurately.

Comparison of PAL and Chain-of-Thought (CoT):

Feature Chain-of-Thought (CoT) Program-Aided

Language Model
(PAL)
Processing Uses step-by-step reasoning in Generates and
Method natural language executes programs
(e.g., Python code)
Computatio Relies solely on LLM's Uses code execution
n language capabilities for mathematical and
logical operations
Accuracy Sometimes prone to errors in More precise for
logic or calculations complex
 computations
Example "The sum of 25 and 47 is... "Execute: print(25
First, add 20 and 40 to get 60, + 47) → Output: 72"
then add the remaining
numbers..."

Example: Solving a Math Problem with PAL

Problem: "What is the sum of the first 100 positive integers?"

 CoT Approach: The model explains the formula: Sum = n(n+1)/2, then manually
applies it.
 PAL Approach: The model generates and runs Python code:

n = 100

sum_n = n * (n + 1) // 2

print(sum_n)

The output is 5050, ensuring accuracy.

Thus, PAL enhances logical correctness by leveraging executable code.

Lifecycle of a Generative AI project

LLM-Powered Applications: Overview & Integration

LLMs (Large Language Models) are powerful AI tools that generate human-like text based
on input prompts. However, for real-world applications, they often require integration with
external data sources and applications to enhance their capabilities and provide more
relevant, up-to-date, and actionable responses.

LLM-Powered Applications

1. Overcoming LLM Limitations with External Integrations

LLMs are trained on vast datasets, but they have limitations such as outdated knowledge and
lack of real-time data access. To overcome this, LLMs can be connected to:

 Databases (e.g., for retrieving structured data)

 APIs (e.g., weather, stock market, or CRM systems)
 Web Sources (e.g., live news, updated documentation)
 Private Documents (e.g., company reports, internal knowledge bases)

2. Integrating LLMs into Applications

To deploy LLMs effectively, they need to be fully integrated within applications. This
involves:

 Passing user input to the LLM

 Receiving and processing the LLM’s response
 Connecting to external components for additional data

3. Role of Orchestration Libraries

To manage communication between the LLM, external data sources, and the application,
developers use orchestration libraries such as:

 LangChain – Helps connect LLMs with APIs, databases, and memory storage.
 LLamaIndex – Useful for retrieving information from external documents.
 Haystack – Enables document search and retrieval for LLM-enhanced applications.

These libraries enable real-time data access, allowing LLMs to interact dynamically with
users and external systems.

LLM-Powered Application Architecture

A typical LLM-powered application follows this structure:

1. User Prompt → The user enters a query.

2. Orchestration Layer → Manages interactions between the LLM and external
sources.
3. External Data Sources → LLM fetches data from documents, databases, APIs, or
web pages.
4. LLM Processing → Generates a response based on user input and external data.
5. Application Response → The final result is returned to the user.

Example Use Cases of LLM-Powered Applications

✔ Customer Support – AI chatbots integrated with CRM systems.

✔ Legal & Compliance – LLMs retrieving case laws from legal databases.
✔ E-commerce & Recommendations – LLMs accessing product catalogues and user
preferences.
✔ Healthcare – LLMs assisting in diagnosis by fetching medical records.

Conclusion
LLMs alone have limitations, but integrating them with external applications, data sources,
and APIs through an orchestration library significantly enhances their functionality. This
approach allows businesses to build more intelligent, data-driven AI applications that
provide real-time, relevant, and accurate responses.

Example:

How It Works (Fast & Local LLM Setup)

1. LLM Used:
 Instead of OpenAI, we use GPT4All or Llama 2 (7B model), which can run
on a regular CPU.
 These models are downloaded and executed locally without an internet
connection.
2. Data Sources (External Integrations):
 A local SQLite database stores transaction details.
 The LLM fetches real-time exchange rates from a simple API.
 It processes and categorizes spending from a CSV file.
3. Orchestration:
 We use LangChain to manage LLM interactions.
 Pandas & Matplotlib help analyze and visualize spending patterns.

Example Flow of the Finance Assistant

User:

"How much did I spend on food last month?"

LLM-Powered Response:

"You spent $420 on food in March 2024. That’s 12% more than February. Would you like
to set a budget limit?"

Fast Implementation (Python Code Example)

from langchain.llms import GPT4All
import sqlite3
import pandas as pd

# Load Local LLM (No OpenAI, Runs on CPU)

llm = GPT4All(model_path="gpt4all-model.bin")

# Connect to local SQLite database

conn = sqlite3.connect("finance.db")
cursor = conn.cursor()

# Fetch spending data

cursor.execute("SELECT SUM(amount) FROM transactions WHERE category='Food'
AND date LIKE '2024-03%'")
food_expense = cursor.fetchone()[0]

# Generate AI response
query = f"My food expense last month was ${food_expense}. How can I reduce
it?"
response = llm(query)

print("Finance Assistant:", response)

Why is This Interesting?

Runs on a normal CPU (no expensive GPUs required!)

Works locally (privacy-friendly, no internet dependency)
Fast processing using a lightweight LLM like GPT4All
Real-world use case (helps users manage finances efficiently)

Components of an LLM Application Architecture:

1. User Interface (UI): Enables users to interact with the LLM-powered application.
2. Orchestration Layer: Manages input processing, API calls, and response generation.
3. LLM Model: The core model that generates text-based outputs.
4. External Data Sources: Databases, APIs, or documents providing relevant
information.
5. Retrieval-Augmented Generation (RAG): Enhances responses by retrieving real-time
data.
6. Security & Monitoring: Ensures safe and optimized interactions.

Interaction with External Applications:

LLMs integrate with APIs, databases, and external tools to enhance their responses. This
connection improves accuracy, updates real-time information, and ensures contextual
relevance.

Example:

A customer support chatbot using an LLM can connect to an e-commerce database via APIs.
When a user asks about order status, the chatbot retrieves real-time tracking details and
provides accurate responses instead of generating a generic reply.