Get up and running with large language models locally.

Download

Download

curl -fsSL https://ollama.com/install.sh | sh

Manual install instructions

The official Ollama Docker image ollama/ollama is available on Docker Hub.

• ollama-python
• ollama-js

To run and chat with Llama 2:

ollama run llama2

Ollama supports a list of models available on ollama.com/library

Here are some example models that can be downloaded:

Note: You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.

Ollama supports importing GGUF models in the Modelfile:

1. Create a file named Modelfile, with a FROM instruction with the local filepath to the model you want to import.

   FROM ./vicuna-33b.Q4_0.gguf
   

2. Create the model in Ollama

   ollama create example -f Modelfile
   

3. Run the model

   ollama run example
   

See the guide on importing models for more information.

Models from the Ollama library can be customized with a prompt. For example, to customize the llama2 model:

ollama pull llama2

Create a Modelfile:

FROM llama2

# set the temperature to 1 [higher is more creative, lower is more coherent]
PARAMETER temperature 1

# set the system message
SYSTEM """
You are Mario from Super Mario Bros. Answer as Mario, the assistant, only.
"""

Next, create and run the model:

ollama create mario -f ./Modelfile
ollama run mario
>>> hi
Hello! It's your friend Mario.

For more examples, see the examples directory. For more information on working with a Modelfile, see the Modelfile documentation.

ollama create is used to create a model from a Modelfile.

ollama create mymodel -f ./Modelfile

ollama pull llama2

This command can also be used to update a local model. Only the diff will be pulled.

ollama rm llama2

ollama cp llama2 my-llama2

For multiline input, you can wrap text with """:

>>> """Hello,
... world!
... """
I'm a basic program that prints the famous "Hello, world!" message to the console.

>>> What's in this image? /Users/jmorgan/Desktop/smile.png
The image features a yellow smiley face, which is likely the central focus of the picture.

$ ollama run llama2 "Summarize this file: $(cat README.md)"
 Ollama is a lightweight, extensible framework for building and running language models on the local machine. It provides a simple API for creating, running, and managing models, as well as a library of pre-built models that can be easily used in a variety of applications.

ollama list

ollama serve is used when you want to start ollama without running the desktop application.

Install cmake and go:

brew install cmake go

Then generate dependencies:

go generate ./...

Then build the binary:

go build .

More detailed instructions can be found in the developer guide

Next, start the server:

./ollama serve

Finally, in a separate shell, run a model:

./ollama run llama2

Ollama has a REST API for running and managing models.

curl http://localhost:11434/api/generate -d '{
  "model": "llama2",
  "prompt":"Why is the sky blue?"
}'

curl http://localhost:11434/api/chat -d '{
  "model": "mistral",
  "messages": [
    { "role": "user", "content": "why is the sky blue?" }
  ]
}'

See the API documentation for all endpoints.

• Bionic GPT
• Enchanted (macOS native)
• HTML UI
• Chatbot UI
• Typescript UI
• Minimalistic React UI for Ollama Models
• Open WebUI
• Ollamac
• big-AGI
• Cheshire Cat assistant framework
• Amica
• chatd
• Ollama-SwiftUI
• MindMac
• NextJS Web Interface for Ollama
• Msty
• Chatbox

• oterm
• Ellama Emacs client
• Emacs client
• gen.nvim
• ollama.nvim
• ollama-chat.nvim
• ogpt.nvim
• gptel Emacs client
• Oatmeal
• cmdh
• tenere
• llm-ollama for Datasette's LLM CLI.
• ShellOracle

• MindsDB

• Pacman
• Helm Chart

• LangChain and LangChain.js with example
• LangChainGo with example
• LangChain4j with example
• LlamaIndex
• LangChain4j
• LiteLLM
• OllamaSharp for .NET
• Ollama for Ruby
• Ollama-rs for Rust
• Ollama4j for Java
• ModelFusion Typescript Library
• OllamaKit for Swift
• Ollama for Dart
• Ollama for Laravel
• LangChainDart
• Semantic Kernel - Python
• Haystack
• Elixir LangChain
• Ollama for R - rollama
• Ollama-ex for Elixir
• Ollama Connector for SAP ABAP

• Enchanted
• Maid

• Raycast extension
• Discollama (Discord bot inside the Ollama discord channel)
• Continue
• Obsidian Ollama plugin
• Logseq Ollama plugin
• Dagger Chatbot
• Discord AI Bot
• Ollama Telegram Bot
• Hass Ollama Conversation
• Rivet plugin
• Llama Coder (Copilot alternative using Ollama)
• Obsidian BMO Chatbot plugin
• Copilot for Obsidian plugin
• Obsidian Local GPT plugin
• Open Interpreter
• twinny (Copilot and Copilot chat alternative using Ollama)
• Wingman-AI (Copilot code and chat alternative using Ollama and HuggingFace)
• Page Assist (Chrome Extension)