2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

Open in app

38
Search

Get unlimited access to the best of Medium for less than $1/week. Become a member

Build Whatsapp Chatbot with Flask and Open

Source LLM — LLAMA3🤖
Mayankchugh Jobathk · Follow
7 min read · May 31, 2024

Listen Share More

Build a WhatsApp chatbot with Flask and open-source LLM (LLAMA3🤖). In this
tutorial, we will create an intelligent WhatsApp chatbot using Flask, OpenAI, and
open-source models from AnyScale, Twilio, and Ngrok. Follow our step-by-step
guide to implement these technologies and discover how they work together to
build an interactive and clear conversational experience, making technology
accessible and engaging.

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 1/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

We will use Ngrok and Flask as a server, Twilio as a WhatsApp agent, an open-source
model using Anyscale API to generate the answer, and Flask app as an API for the
request/response model. 🛠 #Ngrok #Flask #Twilio #AnyscaleAPI #OpenSource #API

But before we go into details, let’s see it in action.

Demo of a simple WhatsApp chatbot prototype using a large language model to

answer questions and generate activity ideas

If you would like to give us feedback, please contact me. What follows here is a
technical overview of how we made this prototype.

🛠️Steps we need to follow;

1. 🆓 Create free account in Anyscale for Open Source LLM, Twilio and Ngrok

👉Firstly, go to AnyScale (https://www.anyscale.com/)and sign up for a free

account, which will give you $50 credit; only email is required; no credit card or
mobile verification is required. Follow the instructions to create an AnyScale API
key. Note down the API key, as you’ll need it in your code.

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 2/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

Next, visit the Twilio website and sign up for a free account.

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 3/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 4/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

After creating your Twilio account, go to the Twilio Console. In the sidebar, locate
and click on. send a WhatsApp message . This connection enables Twilio to send and
receive messages on your behalf via the WhatsApp platform. 📱

Lastly, create a Ngrok account. Ngrok is a free tool that allows us to tunnel from a
public URL to our local application. ✨

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 5/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

Creating an account in Ngrok is important because we need an authentication token

for our project before we can start connecting. 🌟

2. ⚙️Setting Up the Project Environment: Best Practices and Key Configurations

Start by creating a virtual environment using the following commands in your

terminal:

conda create -p venv pyhton=3.8 -y

source activate ./venv

A virtual environment helps isolate project dependencies, ensuring a clean and

organized development environment.
https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 6/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

📋 Define Project Requirements

Specify the packages your project depends on by creating a requirements.txt file.

Install them using:

pip install -r requirements.txt

This step ensures consistent package versions and simplifies environment setup for
collaborators.

🔐 Ngrok Authentication

Add your Ngrok authentication token to your local environment. Execute the
following command:

ngrok authtoken YOUR_AUTH_TOKEN

This command links your local Ngrok client to your Ngrok account, enabling
features like tunnelling your local server to a public URL.

🤐 Secure Secrets &API Key

Prioritize security by storing sensitive information, such as your AnuScale API key,
in a dedicated environment file ( .env ). Create a new file named .env in your
project directory and add the following line:

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 7/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

Add your_secrets_key secrets and actual API key obtained during your AnyScale
(https://www.anyscale.com/) account setup. Using a dedicated environment file
enhances security and maintains a separation between configuration details and
your codebase.

3. 🤖💬 Implementing the Flask App for a WhatsApp Chatbot with open source
model and Twilio

In this section, we’ll look at the implementation details of the Flask application
responsible for handling incoming WhatsApp messages, generating responses using
Open Source Model llama3 using AnyScale, and sending replies with Twilio.

🚀 Initializing Flask and Loading Environment Variables

import openai
from openai import OpenAI
from flask import Flask, request
from twilio.twiml.messaging_response import MessagingResponse
from twilio.rest import Client
from dotenv import load_dotenv
import os
import sys

load_dotenv()
app = Flask(__name__)

The code imports necessary modules, including Flask for building the web
application, Twilio for handling messages, and Open Source LLM Model for utilizing
the llama3 model from AnyScale. The load_dotenv() function loads environment
variables such as Anyscale API Key, from a file named .env .

🔑 Retrieving Secrets and API Key

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 8/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

anyscale_api_key=os.getenv("ANY_SCALE_API_KEY")
anyscale_base_url = os.getenv("ANY_SCALE_API_BASE")
MODEL_NAME = 'meta-llama/Meta-Llama-3-8B-Instruct'

account_sid = os.getenv("TWILIO_ACCOUNT_SID")
auth_token = os.getenv("TWILIO_AUTH_TOKEN")

This line retrieves the AnyScale API key from environment variables. Storing
sensitive information like API keys in environment variables enhances security.

💭 📬 Defining the Flask Route for Incoming Messages and Generate Answers
with LLAMA3 using AnyScale

clientopenAi = OpenAI(
base_url="https://api.endpoints.anyscale.com/v1",
api_key=anyscale_api_key
)

# Handle incoming messages

@app.route('/sms', methods=['POST'])
def handle_incoming_message():

try:

incoming_message = request.form['Body']
print(f"incoming_message:: {incoming_message}")
if incoming_message.lower() == 'restart':
print('Restarting app...')
# You can replace the command with the appropriate command to resta
os.execv(sys.executable, ['python'] + sys.argv)
else:
# Add the new message to the history list
history.append(f"user query: {incoming_message}")

# Remove the oldest message if the history list size is greater tha
if len(history) > 10:
history.pop(0)

# Pass the history list to LLM Model with message indicating it's p
prompt = incoming_message+"\nThese messages below are my previous m
print(prompt)

response = clientopenAi.chat.completions.create(
https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 9/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium
model=MODEL_NAME,

messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": prompt}
]
).choices[0].message.content
print(response)

print("before append history :",history)

# Add the LLM Model llama3 response to the history list
history.append(f"{MODEL_NAME} response: {response}")

print("HISTORY:\n",history)
# Remove the oldest message if the history list size is greater tha
if len(history) > 10:
history.pop(0)

# Create a Twilio MessagingResponse object

twilio_response = MessagingResponse()

# Add the response to the Twilio MessagingResponse object

msg = twilio_response.message()
msg.body(response)

# Send the response back to the user

print("twilio_response :",twilio_response)

# Return the Twilio MessagingResponse object

return str(twilio_response)
except:
pass
return 'OK',200

if __name__ == '__main__':
app.run(debug=True, host='0.0.0.0', port=5000)

This section defines a route /sms that Twilio will use to forward incoming messages.
The function sms extracts the user's question, generates a response using the Open
Source Model llama3, creates a Twilio response, and sends it back.

4. 🚀 Running the Flask App

🏃‍♂️ Run the Flask App

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 10/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

python app.py

Executing this command initiates our Flask application, making it accessible locally
on port 5000.

🌐 Set Up Ngrok

ngrok http 5000

Ngrok allows you to expose your local server to the internet. By running this
command, you create a secure tunnel that links Ngrok’s public URL to your Flask
app running on port 5000.

🌐 Access Your Public URL

After running Ngrok, we’ll receive a public URL (e.g., https://abcd1234.ngrok.io ).

We will use this URL as the webhook for services like Twilio to forward incoming
messages to our local Flask server.

⚙️Configure Twilio with Ngrok URL

Lastly, Copy this public URL, and paste it with the /sms route into the Twilio
Console. Enter this URL into Twilio, and when you open WhatsApp, you can finally
start the conversation.
https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 11/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

5. 📱💬 Initiate WhatsApp Conversation

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 12/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 13/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

Once you’ve completed the integration of your Flask app with Twilio and set up the
necessary configurations, you can start a conversation on WhatsApp with your
Twilio-enabled Twilio Sandbox number.

As soon as you send a message, you’ll experience the seamless interaction with your
Flask app. Twilio will forward the message to your local server, where your chatbot
powered by AnyScale will generate responses.

💭 Notes 💬

Running applications in background:

nohup python app.py

nohup ngrok http 5000

Find the process and kill it

lsof -i:5000

or

ps aux | grep 'python app.py'

Copy the PID of the running process to kill

kill -9 :5000

GitHub - mayankchugh-learning/WhatsApp-Bot-using-GenAI:
Testing-Bot-using-GenAI - Flask==2.1.1…
Testing-Bot-using-GenAI - Flask==2.1.1 twilio==6.50.1
openai==0.28.0 python-dotenv …
github.com

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 14/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

https://www.linkedin.com/in/mchugh77

mayankchugh-learning (Mayank Chugh)

AI & ML Enthusiastic
huggingface.co

🎉 Conclusion

In this tutorial, we walked through the steps of building a WhatsApp chatbot using
Flask, Open Source model llama3 using AnyScale, Twilio, and Ngrok. Integrating
these powerful technologies creates a dynamic, intelligent conversational agent that
responds to real-time user queries. Following these steps, you’ve built a functional
WhatsApp chatbot and gained insights into integrating various technologies to
create an intelligent and responsive conversational system. 🚀🤖💬

Whatsappchatbot Openai Chatgpt OpenAI Flask Twilio

Follow

Written by Mayankchugh Jobathk

44 Followers

More from Mayankchugh Jobathk

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 15/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

Mayankchugh Jobathk

Build a Document-Based RAG Question-Answering App with Ollama and

use Streamlit as a front-end
Welcome to this comprehensive tutorial! Today, I’ll guide you through the process of creating a
document-based question-answering…

Jun 23 1

Mayankchugh Jobathk

Text-to-Image using Stable-Diffusion-3-Medium: Step-by-Step Guide

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 16/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

Stable Diffusion 3 Medium is an advanced text-to-image model known as a Multimodal

Diffusion Transformer (MMDiT). This model delivers…

Jul 9 1

Mayankchugh Jobathk

Multimodal Retrieval-augmented generation (MMRAG)

Introduction

Jul 23 1

Mayankchugh Jobathk

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 17/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

The Evolution of Large Language Models (LLMs): A Journey from GPT to

GPT-4O
The field of artificial intelligence has witnessed unprecedented growth, particularly with the
advent of large language models (LLMs)…

Jul 18

See all from Mayankchugh Jobathk

Recommended from Medium

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 18/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

Anoop Maurya in Towards AI

Building Your Own Generative Search Engine for Local Files Using Open-
Source Models 🧐📂:Part-2
Hey fellow AI nerds! 🤓👋 Thanks for showing all the love on the first article — it was as
heartwarming as finding out your code works…

6d ago 423 2

Juan Acosta in FAUN — Developer Community 🐾

Integrating Twilio with Flask: Building the Backbone of a WhatsApp Real

Estate Chatbot
https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 19/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

Part 2 of the Series: Building a Smart WhatsApp Real Estate Chatbot: AI, Twilio, BERT, and
LangChain Integration

Sep 25

Lists

AI Regulation
6 stories · 597 saves

Natural Language Processing

1777 stories · 1376 saves

What is ChatGPT?
9 stories · 455 saves

data science and AI

40 stories · 271 saves

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 20/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

Alisha Saboowala

WhatsApp Your RAG Application

Build a RAG chatbot using the OpenAI Assistants tool with a WhatsApp interface

Jul 25 1

C. C. Python Programming

Building a Notification Service with FastAPI and Python

FastAPI is a modern, fast web framework used to build APIs with Python. It is designed to help
developers create applications efficiently…

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 21/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

4d ago 58 1

Devvrat Rana

Understanding LangChain Agents: A Beginner’s Guide to How LangChain

Agents Work
Introduction:

Jun 2 66

Vishal Rajput in AIGuys

The Prompt Report: Prompt Engineering Techniques

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 22/23
2024/10/27 上午11:56 Build Whatsapp Chatbot with Flask and Open Source LLM — LLAMA3🤖 | by Mayankchugh Jobathk | Medium

Prompting Techniques Survey

Oct 14 904 9

See more recommendations

https://medium.com/@mayankchugh.jobathk/build-whatsapp-chatbot-with-flask-and-open-source-llm-llama3-0284619a96d4 23/23