One-Stop Framework Building Applications with LLMs

BE G I NNE R C HAT G PT G E NE RAT I VE A I LLM S M A C HI NE LE A RNI NG PYT HO N

Introduction

Large Language Models (LLMs) have been gaining popularity for the past few years. And with the entry of
Open AIs ChatGPT, there was a massive popularity gain in the Industry towards these LLMs. These Large
Language Models are being worked upon to create different applications from Question Answering
Chatbots to Text Generators to Conversational Bots and much more. Many new APIs are being created and
each has its own way of training and testing our own data to it. This is where LangChain fits in, a Python
Framework/Library for developing applications powered with these Language Models. In this article, we
will be going to be building applications with LLMs.

Learning Objectives

To understand the LangChain Framework

To create applications with LangChain
Understanding the Components involved in it
Building Chains through LangChain for Language Models
Understanding Agents and Prompts in LangChain

This article was published as a part of the Data Science Blogathon.

What is LangChain? Why is it necessary?

LangChain is a Python Library for building applications powered by Large Language Models. It not only
takes care of connecting to different Large Language Models through APIs but even makes these LLMs
connect to a data source and even makes them aware of their environment and interact with it. So how
does it fit in? The thing is these Large Language Models on their own in isolation may not be that powerful.
Thus with LangChain, we can connect them with external data sources and computation, which will greatly
help them to come up with good answers.

LangChain thus makes it possible for us to make the Language Models connect to our very own database
and create applications around them, allowing them to reference the data. With LangChain, not only you
can create Language Models to create Question Answer Bots based on your data provided, but even make
the Language Models take certain actions based on questions thus making them Data-Aware and Agentic.
We will be looking into these actions further down in the article.

LangChain Framework contains Components that constitute LLM wrappers, which are wrappers for popular
language model APIs, from Open AI, Hugging Face, etc. It even includes the Prompt-Templates for creating
our own Prompts. LangChain, the name itself has the word Chain, thus making it possible to chain multiple
Components together and finally Agents, which we have talked about before, that allow the model to
interact with external data and computations.
Installing LangChain

Like all other Python libraries, LangChain can be installed through Python’s pip command. The command
for this is:

pip install -qU langchain

This will download and install the latest stable version of langchain Framework in Python. The langchain
Framework comes with many LLM wrappers, ChatBot Wrappers, Chat Schemas, and Prompt-Templates

Apart from LangChain Packages, we need to even install the following packages, which we will be working
on in this article

pip install ai21 pip install -qU huggingface_hub pip install -qU openai

This will install the hugging face hub, where we will be able to work with hugging face APIs. OpenAI is
installed, thus we can work with GPT models, and one more LLM model we have installed is the ai21, which
is the Studio AI21 Language Model. We will be working with these 3 models, writing examples, and
understanding how LangChain fits in.

Building Applications with LLMs Wrapper

LangChain provides wrappers to different models from LLMs to Chat Models to Text Embedding Models.
Large Language Models are the ones that take text for the input and the output returned is a text string. In
this section, we will take a look at the LangChain wrappers i.e. the standard interface it provides for
different Large Language Models like Hugging Face, Open AI, Studio AI21, etc.

Let’s start by using the Open AI model in LangChain. The respective code will be

# importing Open AI Wrapper from LangChain from langchain.llms import OpenAI # provide your API KEY here

os.environ["OPENAI_API_KEY"] = 'Your OpenAI API KEY' # initializing OpenAI LLM llm = OpenAI(model_name="text-

ada-001") # query query = 'Tell me a joke' # model output print(llm(query))

Here we import the OpenAI Wrapper from LangChain. Then we create an Instance of it named llm and
provide input, the model name we want to use. Now to this llm variable, we directly pass the query to get
an output. Running this code has given the output:

We see that the model gives out a joke based on the query provided. Now let’s try this with the Hugging
Face Model.

# importing Hugging Face Wrapper from LangChain from langchain import HuggingFaceHub # provide your API KEY

here os.environ['HUGGINGFACEHUB_API_TOKEN'] = 'Your Hugging Face API Token' # initialize Hugging Face LLM

flan_t5_model = HuggingFaceHub( repo_id="google/flan-t5-xxl", model_kwargs={"temperature":1e-1} ) query1 =

"Who was the first person to go to Space?" query2 = "What is 2 + 2 equals to?" generate =

flan_t5_model.generate([query1, query2]) print(generate.generations)

We have followed the same process that we have followed with the OpenAI model. Here we are working
with the flan-t5 model and have set the temperature to 1e-1. The only difference is that here we use the
generate() method of the LLM model to pass the queries. This generate() method is used when you want to
pass multiple queries at the same time. These are passed in the form of a list, which we have done so in
the above. To get the model output, we then call the generations function. The resulting output we get
after we run the code is

We see 2 answers from the model. The first is Yuri Gagarin, who was indeed the first person to enter the
space and the second answer is 4, which is right. Finally, we will look at another LLM wrapper, this time for
the AI21 Studio which provides us with the API access to Jurrasic – 2 LLM. Let’s look at the code:

# importing AI21 Studio Wrapper from LangChain from langchain.llms import AI21 # provide your API KEY here

os.environ['AI21_API_KEY'] = 'Your API KEY' # initializing OpenAI LLM llm = AI21() # query query = 'Tell me a

joke on Cars' # model output print(llm(query))

Again, we see that the overall code implementation is almost similar to the OpenAI example that we have
seen earlier. The only difference is the API Key and the LLM wrapper that is being imported which is the
AI21. Here we have given a query to tell a joke on cars, let’s see how it performs

The AI21 really did a fab job with the answer it provided. So in this section, we have learned, how to use
different LLM wrappers from LangChain for working with different Language Models. Apart from these 3
wrappers that we have seen, LangChain provides many more wrappers to many other Language Models out
there.

Prompt Templates

Prompts are a key part when developing applications with Large Language Models. Initially, one would
have to retrain the entire model or have to work with completely a different model to do different tasks, like
one model for Translation, and one model for Summarization. But the entry of Prompt Templates has
changed it all. With Prompt templates, we can make the Language Model do anything from Translation to
Question Answering, to Text Generation/Summarization on different data.

In this article, we will take a look into Prompt Templates in langchain with the OpenAI models. Let’s begin
by creating a template and then giving it to LangChain’s PromptTemplate class

from langchain import PromptTemplate # creating a Prompt Template template = """The following is a

conversation between a time traveler and a historian. The time traveler is from the future and tends to make
humorous comparisons between past and future events: Historian: {query} Time Traveler: """ # assigning the

template to the PromptTemplate Class prompt = PromptTemplate( input_variables=["query"], template=template )

The first step is to write a template. Here we have written a template stating that we want the AI to act like
a funny Time Traveller, that is we have set a context for the AI, and how it should be acting. Then we
provide the query considering we are a historian. The query is present in {} because we want that part to be
replaced by the question we want to ask. Then this template is given to the PromptTemplate class from
LangChain. Here we pass the template to the template variable, and we then tell the input variable as
“query” (the input variable is the place where our questions sit).

Let’s try to create a question and observer the Prompt that will be generated.

# printing an example Prompt print(prompt.format(query='Are there flying cars?'))

Here in the output, we see that {query} is replaced by the question that we have given to the “query”
variable that was given to the format() function of the Prompt Instance. So the PromptTemplate does the
job of formatting our query before we can send it to the model. Now, we have successfully created a
Prompt Template. We will now test this template with an OpenAI model. The following will be the code:

# creating the llm wrapper llm = OpenAI(model_name="text-davinci-003",temperature=1) # model's output to the

query print( llm(prompt.format( query='Are there flying cars?') ) )

Here again, we are working with the LLM wrapper from langchain for OpenAI Language Model. We are then
giving the Prompt directly to the llm, just like how we have passed in the beginning code example. This will
then print the output generated by the model, which is:

Output

Ha! No way, although I did see a farmer the other day using a robotic scarecrow to shoo away birds from
his cornfield. Does that count?

Seeing the output, we can say that the Language model did indeed act like a Time Traveller. Prompt
Templates can be provided with multiple queries. Let’s try to create a template to provide multiple inputs to
the Language Model

multi_query_template = """Answer the following questions one at a time. Questions: {questions} Answers: """
long_prompt = PromptTemplate( template=multi_query_template, input_variables=["questions"] ) qs = [ "Which

IPL team won the IPL in the 2016 season?", "How many Kilometers is 20 Miles?", "How many legs does a lemon

leaf have?" ] print( llm(prompt.format( query=qs) ) )

The above depicts the code for multiple query templates. Here in the Prompt Template, we have written at
the start, that the questions must be answered one at a time. Then we provide this template to the
PromptTemplate class. Store all the questions in a list called qs. Just like before, we pass this qs list to
the query variable in the prompt.format() function and give it to the model. Now let’s check the output
produced by the Language Model
We see that the OpenAI Language Model has given all the right output answers one by one. So far, what we
have done with Prompt Template is just a glimpse. We can do much more using the Prompt Components
from the LangChain Framework. One example is we can perform Few Shot Learning by providing some
examples in the context of the Prompt Template itself.

LangChain – Chaining Components Together

The LangChain, the name itself has the word Chain in it. In this section, we will look at the Chain Module of
LangChain. When developing applications, using Language Models in isolation is fine, i.e. for small
applications, but when creating complex ones, it’s better to chain LLMs, i.e. either chaining two similar
LLMs or two different ones. LangChain provides a Standard Interface for chaining different Language
Models together. Chains can be used to connect different Components like LLMs and even Prompts
together to perform a particular action

Example

Let’s start by taking a simple example of chaining an OpenAI Language Model with a Prompt.

from langchain.llms import OpenAI from langchain import PromptTemplate, LLMChain # creating a Prompt Template

template = """The following is a conversation between a human and an AI Assistant. Whatever the human asks to

explain, the AI assistant explains it with humour and by taking banana as an example Human: {query} AI: """ #

assigning the template to the PromptTemplate Class prompt = PromptTemplate( input_variables=["query"],

template=template ) # query query = "Explain Machine Learning?" # creating an llm chain llm =

OpenAI(model_name="text-davinci-003",temperature=1) llm_chain = LLMChain(prompt=prompt, llm=llm) # model

output print(llm_chain.run(query))

Create a template, saying that every question asked by the user is answered by the AI in a funny way
and taking an example of a banana.
Then this template passed to the PromptTemplate() function thus creating a Prompt Template out of it.
Then with the OpenAI LLM wrapper, we initialize the OpenAI model.
We have imported the LLMChain from LangChain. LLMChain is one of the Simplest Chains provided by
LangChain. To this, we pass the Prompt Template and the Language Model
Finally, we pass the query directly to the llm_chain’s run() function

What LLMChain does is, first it sends the input query to the first element in the chain, i.e. to the Prompt,
i.e. the PromptTemplate. Here the input gets formatted to a particular Prompt. This formatted Prompt is
then passed to the next element in the chain, i.e. the language model. So a chain can be considered like a
Pipeline. Let’s see the output generated.

Output: Machine Learning is like a banana. You keep giving it data and it slowly develops the skills it needs
to make more intelligent decisions. It’s like a banana growing from a little green fruit to a fully ripe and
delicious one. With Machine Learning, the more data you give it, the smarter it gets!

Here we do get an answer to the question we have asked. The answer generated is both funny and even
contains the banana example in it. So we might get a question now, can two chains be chained together?
Well, the answer is absolutely YES. In the next example, we will be doing exactly the same. We will create
two chains with two models, then we will chain them both together. The code for this is

from langchain.llms import OpenAI from langchain.chains import LLMChain from langchain.prompts import
PromptTemplate from langchain.chains import SimpleSequentialChain # creating the first template

first_template = """Given the IPL Team Name, tell the Year in which they first won the trophy. % IPL TEAM
NAME {team_name} YOUR RESPONSE: """ team_template = PromptTemplate(input_variables=["team_name"],

template=first_template) # creating the team_chain that holds the year informatino team_chain =
LLMChain(llm=llm, prompt=team_template) # creating the second Template second_template = """Given the Year,

name the Highest Scoring Batsman in the IPL for that Year. % YEAR {year} YOUR RESPONSE: """ batsman_template
= PromptTemplate(input_variables=["year"], template=second_template) # creating the batsman_chain that holds

the bastman information batsman_chain = LLMChain(llm=llm, prompt=batsman_template) # combining two LLMChains

final_chain = SimpleSequentialChain(chains=[team_chain, batsman_chain], verbose=True) # checking the chain

output final_output = final_chain.run("Sunrisers Hyderabad")

Firstly, we have created two templates, the template one asks the model for the year in which a
particular Cricket Team won, and the second template, given the year, tells the highest-scoring
batsman
After creating the templates, then we create the PromptTemplates for both of them
Then we create our first chain, the team_chain, which contains our OpenAI Language Model, which we
have defined in the first code and the first template. The input to our first template is the TEAM NAME
Then we create the second chain. This chain takes the same model but the template given is the
second template. This chain takes in the YEAR and gives the highest-scoring IPL batsman in that year.
Finally, we then combine these two chains with the SimpleSequentialChain() function and store it in the
final_chain variable. Here we pass the chains in the form of a list. It’s important to make sure that the
chains are stored in the list in the same order that they need to be. We even set the verbose to True, so
we can better understand the output

Now we have run the code by giving Sunrisers Hyderabad for the input to our final chain. The output
returned was:

The output produced is indeed True. So how did it do it? Firstly, the input Sunrisers Hyderabad is fed to the
first chain i.e. the team_chain. The output of the team_chain is in line 1. So it returned the output. i.e. the
year 2016. The team_chain generates an output, which serves as the input for the second chain, the
batsman_chain. The batsman_chain takes the input of the Year, specifically 2016, and produces the name
of the highest-scoring IPL batsman in that year. The second line displays the output of this chain. So this
is how chaining/combination of two or more chains work in particular.

Again, we have just looked at only some of the Chaining concepts in LangChain. Developers can work with
the Chains to create various applications. They can also categorize the chains themselves, although this
topic is beyond the scope of this article.

Agents in LangChain
Despite their immense power, Large Language Models often lack basic functionalities such as logic and
calculation. They can struggle with simple calculations that even small calculator programs can handle
more effectively.

Agents, on the other hand, have access to tools and toolkits that enable them to perform specific actions.
For instance, the Python Agent utilizes the PythonREPLTool to execute Python commands. The Large
Language Model provides instructions to the agent on what code to run.

Example

from langchain.agents.agent_toolkits import create_python_agent from langchain.tools.python.tool import

PythonREPLTool from langchain.python import PythonREPL from langchain.llms.openai import OpenAI # creating

Python agent agent_executor = create_python_agent( llm=OpenAI(temperature=0, max_tokens=1000),

tool=PythonREPLTool(), verbose=True ) agent_executor.run("What is 1.12 raised to the power 1.19?")

Here to create the Python agent, we use the create_python_agent() object from the langchain. To this, we
pass our OpenAI Language Model. And the tool we will work is the PythonREPLTool() which is capable of
running Python code. To get a detailed output, we set the verbose to True. Finally, we run the model with
the input to find the power of 1.12 raised to 1.19. The output generated is

In this process, the language model generates Python code, which is then executed by the agent using the
PythonREPLTool(). The resulting answer is returned by the language model. Agents go beyond code
execution and can also search Google for answers when the language model fails. These powerful
components in the LangChain enable the creation of complex models with high accuracy.

Conclusion

LangChain is a newly developed Python Framework for building applications with powerful Language
Models. LangChain proved a standard interface to work with multiple language models. The Components
in LangChain like the Prompts, Chains, and agents can be worked with to generate powerful applications.
We have learned how to modify Prompts to make the Language Models produce different outputs. And also
looked into how to chain different language models too. Even worked with Agents, that were capable of
running Python Code, which Large Language Models cannot.

Some of the key takeaways from this article include:

The support for different models in LangChain makes it easy to work with different Language Models.
Create agents with LangChain to perform tasks that simple Language Models cannot
 Multiple queries can be fed into the Language Models by changing the Prompt Template.
Combine One or more Language Models together with Chain Components to get accurate answers.
With LangChain, the time to change from one Language Model to another is greatly reduced.

The media shown in this ar ticle is not owned by Analytics Vidhya and is used at the Author’s discretion.

Article Url - https://www.analyticsvidhya.com/blog/2023/05/one-stop-framework-building-applications-

with-llms/

Ajay Kumar Reddy