The LangChain

ecosystem
D E V E L O P I N G L L M A P P L I C AT I O N S W I T H L A N G C H A I N

Jonathan Bennion
AI Engineer & LangChain Contributor
Meet your instructor...

Jonathan Bennion, AI Engineer

ML & AI at Facebook, Google, Amazon,

Disney, EA

Created Logical Fallacy chain in LangChain

Contributor to DeepEval

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

 The state of Large Language Models (LLMs)

1 https://arxiv.org/pdf/2303.18223.pdf

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

What is LangChain?

Open-source framework for connecting:

Large Language Models (LLMs)

Data sources

Other functionality under a unified

syntax

Allows for scalability

Contains modular components

Supports Python and JavaScript

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

Core components of LangChain

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

Hugging Face
Open-source repository of models,
datasets, and tools

Creating a Hugging Face API key:

1. Sign up for a Hugging Face account

2. Navigate to
https://huggingface.co/settings/tokens

3. Select New token and copy the key

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

Hugging Face (Falcon-7b): OpenAI ( gpt-3.5-turbo-instruct ):

from langchain_huggingface import HuggingFaceEndpoint from langchain_openai import OpenAI

llm = HuggingFaceEndpoint( llm = OpenAI(

repo_id='tiiuae/falcon-7b-instruct', model="gpt-3.5-turbo-instruct",
huggingfacehub_api_token=huggingfacehub_api_token api_key=openai_api_key
) )

question = 'Can you still have fun'

output = llm.invoke(question) question = 'Can you still have fun'
output = llm.invoke(question)
print(output)
print(output)

in the rain?
Yes, you can still have fun in the without spending a lot of money?
rain! There are plenty of
Yes, you can still have fun without
spending a lot of money. You could do
activities like hiking, biking, playing
sports, going to the beach, camping...

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

Real-world usage

Examples:

Natural language conversations with

documents

Automate tasks

Data analysis

Note: course uses langchain==0.3.13

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

 Let's practice!
D E V E L O P I N G L L M A P P L I C AT I O N S W I T H L A N G C H A I N
 Prompting strategies
for chatbots
D E V E L O P I N G L L M A P P L I C AT I O N S W I T H L A N G C H A I N

Jonathan Bennion
AI Engineer & LangChain Contributor
 Finding the right model

Hugging Face is a searchable hub for chat

models

Fine-tuned models for more domain-

specific use cases

1 https://huggingface.co/models?pipeline_tag=question-answering&sort=trending

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

Prompt templates
Recipes for generating prompts
Flexible and modular
Can contain: instructions, examples, and additional context

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

Prompt templates
from langchain_core.prompts import PromptTemplate

template = "You are an artificial intelligence assistant, answer the question. {question}"
prompt_template = PromptTemplate(template=template, input_variables=["question"])

print(prompt_template.invoke({"question": "What is LangChain?"}))

text='You are an artificial intelligence assistant, answer the question. What is LangChain?'

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

Integrating PromptTemplate with LLMs
from langchain_huggingface import HuggingFaceEndpoint

llm = HuggingFaceEndpoint(repo_id='tiiuae/falcon-7b-instruct', huggingfacehub_api_token=huggingfacehub_api_toke

llm_chain = prompt_template | llm

question = "What is LangChain?"

print(llm_chain.invoke({"question": question}))

LangChain is an artificial intelligence language model that uses a neural network to generate human-like text

LangChain Expression Language (LCEL)

Chain: connect calls to different components

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

Chat models
from langchain_core.prompts import ChatPromptTemplate

prompt_template = ChatPromptTemplate.from_messages(
[
("system", "You are soto zen master Roshi."),
("human", "What is the essence of Zen?"),
("ai", "When you are hungry, eat. When you are tired, sleep.")
("human", "Respond to the question: {question}")
]
)

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

Integrating ChatPromptTemplate
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini", api_key=openai_api_key)

llm_chain = prompt_template | llm

question='What is the sound of one hand clapping?'

response = llm_chain.invoke({"question": question})

print(response.content)

The sound of one hand clapping is not something that can be easily explained or
understood through words alone. It is a question that has been pondered by Zen
practitioners for centuries, and its purpose is to provoke a deeper inquiry into
the nature of reality and the self. In Zen practice, we often engage...

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

 Let's practice!
D E V E L O P I N G L L M A P P L I C AT I O N S W I T H L A N G C H A I N
 Few-shot prompting
D E V E L O P I N G L L M A P P L I C AT I O N S W I T H L A N G C H A I N

Jonathan Bennion
AI Engineer & LangChain Contributor
Limitations of standard prompt templates

PromptTemplate + ChatPromptTemplate
examples = [
Handling small numbers of examples {
"question": "..."
Don't scale to larger numbers
"answer": "..."
FewShotPromptTemplate
},
...
]

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

Building an example set
examples = [
{
"question": "Does Henry Campbell have any pets?",
"answer": "Henry Campbell has a dog called Pluto."
},
...
]

# Convert DataFrame to list of dicts

examples = df.to_dict(orient="records")

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

Formatting the examples
from langchain_core.prompts import FewShotPromptTemplate, PromptTemplate

example_prompt = PromptTemplate.from_template("Question: {question}\n{answer}")

prompt = example_prompt.invoke({"question": "What is the capital of Italy?"

"answer": "Rome"})
print(prompt.text)

Question: What is the capital of Italy?

Rome

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

FewShotPromptTemplate

prompt_template = FewShotPromptTemplate(
examples=examples,
examples : the list of dicts
example_prompt=example_prompt,
suffix="Question: {input}", example_prompt : formatted template
input_variables=["input"] suffix : suffix to add to the input
)
input_variables

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

Invoking the few-shot prompt template
prompt = prompt_template.invoke({"input": "What is the name of Henry Campbell's dog?"})
print(prompt.text)

Question: Does Henry Campbell have any pets?

Henry Campbell has a dog called Pluto.
...

Question: What is the name of Henry Campbell's dog?

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

Integration with a chain
llm = ChatOpenAI(model="gpt-4o-mini", api_key="...")

llm_chain = prompt_template | llm

response = llm_chain.invoke({"input": "What is the name of Henry Campbell's dog?"})
print(response.content)

The name of Henry Campbell's dog is Pluto.

DEVELOPING LLM APPLICATIONS WITH LANGCHAIN

 Let's practice!
D E V E L O P I N G L L M A P P L I C AT I O N S W I T H L A N G C H A I N