Basic Details
Basic Details
Project Information
● This is your chance to provide a description of the project idea. What are the major
parts of the project? Use flowcharts, diagrams and mockups as much as possible. If
your project involves writing APIs, what might those APIs look like?
○ As mentioned in the ideas list , this project builds upon the work done last
year where an AI chatbot was developed capable of answering questions and
assisting users in building models with gprMax. Currently, the project utilizes
the ChatGPT API.However it incurs a significant cost and is closed source .
This project aims to integrate an open source LLM and remove the need of
buying tokens . The LLM should also be computationally cheap so that the
majority of people can use it. We can also use a service like Open Router
which has a generous free tier and would provide us with access to various
LLMs which would remove the need for the model to be lightweight.
○ Also the previous chatbot was deployed using streamlit , I was thinking about
changing it to Flask/Fast-API as they provide better performance and
scalability compared to streamlit. The new deployment will feature a simple
user interface with just a few API endpoints for core functionality. We can
even add it to a website as a widget if needed.
○ This project also includes fine tuning the LLM on the dataset of gprMax
commands so that it is capable of building models automatically based on
simple instructions.
○ The Major Components include integrating an open source LLM (llama
,mistral or deepseek) , creating a backend api for deployment and fine
tuning. There will also be several optimization steps but these are the major
components for this project.
● Discuss your assumptions
○ I am assuming that I’ll have access to computational resources (GPU’s)
required for fine tuning the model . Google Colab’s free tier does have a GPU
option but we might run into some limitations like GPU availability and vram
bottlenecks . There are some techniques we can use to reduce the amount of
vram used such as QLoRa and memory management(low batch sizes).
○ I am also assuming that a basic UI with just a few api endpoints will be
sufficient for users . Additional features can be added if we find them
necessary .
● Mention your deliverables - break down the bigger picture into smaller tasks and
explain what these might be.
1. LLM Integration
a. Evaluate and choose an open source model that we can use for the
chat bot
b. Modify the previous implementation so that it uses the open source
model instead of using Chat GPT
c. Do some testing to ensure that everything works smoothly and resolve
bugs if needed.
2. Fine Tuning
a. Preprocess the prepared dataset of gprMax commands .
b. Fine Tune the selected model
c. Testing and validating the fine tuned model’s performance .
3. API Development
a. Create and implement a simple API structure with Flask or Fast API .
b. Work on a basic user interface for testing purposes.
c. Test everything so that it works as intended.
4. Deployment
a. Deploy the application in a testing environment and conduct tests
b. Fix bugs or make any changes if needed.
5. Optimization
a. Implement optimization techniques to improve performance and
reduce response time for the chatbot.
6. Documentation
a. Write proper documentation and integration processes for future
contributors/users
b. Include examples and a getting started guide in the documentation for
new users.
● If you have found existing work to build on, mention it, e.g. if you plan to use a
specific algorithm/layer/model/library/framework.
○ Well I’ll be building the chatbot from the previous year’s project. I plan to use
several tools and frameworks for this project. For the model I plan to go with
Llama,mistral or deepseek. We can either provide the option to host the
models locally (which would force us to go with something lightweight) or go
with some service like Open Router