How to Run Open Source LLMs Locally Using Ollama https://www.freecodecamp.org/news/how-to-run-open-source-llms-l...

Forum Donate

Learn to code — free 3,000-hour curriculum

APRIL 2, 2024 / #LLMS

How to Run Open Source

LLMs Locally Using Ollama
Sahil Mahapatra

This article will guide you through downloading and

using Ollama, a powerful tool for interacting with
open-source large language models (LLMs) on your
local machine.

1 de 7 03/04/2024, 09:19 p. m.
How to Run Open Source LLMs Locally Using Ollama https://www.freecodecamp.org/news/how-to-run-open-source-llms-l...

Unlike closed-source models like ChatGPT, Ollama offers

Forum Donate
transparency and customization, making it a valuable resource for
Learn to code — free 3,000-hour curriculum
developers and enthusiasts.

We'll explore how to download Ollama and interact with two

exciting open-source LLM models: LLaMA 2, a text-based model
from Meta, and LLaVA, a multimodal model that can handle both
text and images.

How to Download Ollama

To download Ollama, head on to the of�cial website of Ollama and
hit the download button.

ollama homepage

Ollama supports 3 different operating systems, and the Windows

version is in preview mode.

2 de 7 03/04/2024, 09:19 p. m.
How to Run Open Source LLMs Locally Using Ollama https://www.freecodecamp.org/news/how-to-run-open-source-llms-l...

Forum Donate

Learn to code — free 3,000-hour curriculum

ollama download page

You can choose the executable �le according to your OS and after
successfully downloading the executable �le, you can install it by
running the executable �le.

For Linux users, you have to execute the command that is being
shown on the screen instead of downloading an executable �le.

How to Run Ollama

To show you the power of using open source LLMs locally, I'll
present multiple examples with different open source models with
different use-cases. This will help you to use any future open source
LLM models with ease.

So, lets get started with the �rst example!

How to Run the LLama2 Model from Meta

Llama 2 model is an open-source LLM model from Meta and we'll
interact with it like we'd do with ChatGPT (free version), only text
based interaction.

First, let's download the model using the following command:

ollama run llama2

You should see a output similar to this after the downloading of the
model.

pulling manifest

3 de 7 03/04/2024, 09:19 p. m.
How to Run Open Source LLMs Locally Using Ollama https://www.freecodecamp.org/news/how-to-run-open-source-llms-l...

pulling 8934d96d3f08... 100% ▕█████████████████████████████████████████████████████████

Forum Donate
pulling 8c17c2ebb0ea... 100% ▕█████████████████████████████████████████████████████████
pulling 7c23fb36d801... 100% ▕█████████████████████████████████████████████████████████
Learn to code — free 3,000-hour curriculum
pulling 2e0493f67d0c... 100% ▕█████████████████████████████████████████████████████████
pulling fa304d675061... 100% ▕█████████████████████████████████████████████████████████
pulling 42ba7f8a01dd... 100% ▕█████████████████████████████████████████████████████████
verifying sha256 digest
writing manifest
removing any unused layers
success
>>> Send a message (/? for help)

Now, its should ask you to send a message or prompt. Let's ask
llama2 "What can you do for me?" (You can ask whatever you want).

If you asked the same question as above, then you should get a reply
like this:

>>> What can you do for me?

As a responsible AI language model, I am here to assist you with any questions or tasks
with:

1. Answering questions: I can provide information on a wide range of topics, from scien
2. Generating ideas: I can help you brainstorm ideas for creative projects, or provide
3. Writing assistance: I can help you with writing tasks such as proofreading, editing,
4. Translation: I can translate text from one language to another.
5. Summarizing content: I can summarize long pieces of text, such as articles or docume
6. Creativity: I can help you generate creative ideas for stories, poems, or other form
7. Language learning: I can assist you in learning a new language by providing grammar
8. Chatting: I'm here to chat with you and provide a response to any question or topic

Please let me know if there is anything specific you would like me to help

>>> Send a message (/? for help)

So that is the response that I got from llama2.

To exit the program, you can type ∕exit .

Let's now run a multi-modal model where you can send an image
and ask questions based on that.

How to Run the LLaVA Model

LLaVA is a open-source multi-modal LLM model. A multi-modal

4 de 7 03/04/2024, 09:19 p. m.
How to Run Open Source LLMs Locally Using Ollama https://www.freecodecamp.org/news/how-to-run-open-source-llms-l...

model can take input of multiple types and generate a Forum

response Donate
accordingly.
Learn to code — free 3,000-hour curriculum

Using this model, we are now going to pass an image and ask a
question based on that.

So, �rst things �rst, lets download the model:

ollama run llava

After successfully downloading the model, you should see

something like this in the terminal:

pulling manifest
pulling 170370233dd5... 100% ▕█████████████████████████████████████████████████████████
pulling 72d6f08a42f6... 100% ▕█████████████████████████████████████████████████████████
pulling 43070e2d4e53... 100% ▕█████████████████████████████████████████████████████████
pulling c43332387573... 100% ▕█████████████████████████████████████████████████████████
pulling ed11eda7790d... 100% ▕█████████████████████████████████████████████████████████
pulling 7c658f9561e5... 100% ▕█████████████████████████████████████████████████████████
verifying sha256 digest
writing manifest
removing any unused layers
success
>>> Send a message (/? for help)

I'll be using this image from pexels.com.

This is the output I got from LLaVA:

>>> What's in this image? ./Downloads/test-image-for-llava.jpeg

Added image './Downloads/test-image-for-llava.jpeg'
The image shows a person walking across a crosswalk at an intersection. There are traf
side. The road is marked with lane markings and a pedestrian crossing signal. The area
structures in the immediate vicinity of the person.

>>> Send a message (/? for help)

You can see that its an accurate explanation of the image.

5 de 7 03/04/2024, 09:19 p. m.
How to Run Open Source LLMs Locally Using Ollama https://www.freecodecamp.org/news/how-to-run-open-source-llms-l...

Feel free to try something else and have fun with it. Forum Donate

Learn to code — free 3,000-hour curriculum

Conclusion
That's it! With Ollama, you can experiment with powerful LLMs like
LLaMA 2 and LLaVA on your own machine.

Download Ollama and explore the exciting world of open-source

large language models!

Sahil Mahapatra
Read more posts.

Learn to code for free. freeCodeCamp's open source curriculum has

helped more than 40,000 people get jobs as developers.
Get started

freeCodeCamp is a donor-supported tax-exempt 501(c)(3) charity organization (United States

Federal Tax Identi�cation Number: 82-0779546)

Our mission: to help people learn to code for free. We accomplish this by creating thousands of
videos, articles, and interactive coding lessons - all freely available to the public.

Donations to freeCodeCamp go toward our education initiatives, and help pay for servers,
services, and staff.

You can make a tax-deductible donation here .

6 de 7 03/04/2024, 09:19 p. m.
How to Run Open Source LLMs Locally Using Ollama https://www.freecodecamp.org/news/how-to-run-open-source-llms-l...

Forum Donate
Trending Guides
Learn to code — free 3,000-hour curriculum
Date Formatting in JS Java Iterator Hashmap Cancel a Merge in Git
What is a Linked List? Install Java in Ubuntu Python Ternary Operator
Full Stack Career Guide Python Sort Dict by Key Smart Quotes Copy/Paste
JavaScript Array Length Sets in Python Kotlin vs Java
SQL Temp Table HTML Form Basics Comments in YAML
Pandas Count Rows Python End Program Python XOR Operator
Python Dict Has Key Python List to String Exit Function in Python
String to Array in Java Python Import from File Parse a String in Python
Python Merge Dictionaries Copy a Directory in Linux Reactive Programming Guide
Center Text Vertically CSS What’s a Greedy Algorithm? Edit Commit Messages in Git

Mobile App

Our Charity

About Alumni Network Open Source Shop Support Sponsors Academic Honesty

Code of Conduct Privacy Policy Terms of Service Copyright Policy

7 de 7 03/04/2024, 09:19 p. m.