Cross-platform wrapper scripts for llama-server that simplify the process of listing and running .gguf models with comprehensive configuration management through environment variables and/or configuration files.

Available Scripts:

• llms.ps1 - PowerShell script for Windows, Linux, and macOS (PowerShell Core)
• llms - Bash script for UNIX systems (Linux, macOS, WSL)

This script serves as an intelligent wrapper around llama-server that:

• Simplifies model management: Automatically discovers and runs GGUF models from configured directories
• Provides flexible configuration: Supports both environment variables and INI file for all parameters
• Enables partial matching: Find models using partial names without typing full filenames
• Handles multi-modal models: Automatically detects and loads companion .mmproj-*.gguf files
• Offers dry-run capability: Preview commands before execution
• Maintains consistent defaults: Fallback values ensure the script works out-of-the-box

• Cross-platform support: PowerShell script for Windows, bash script for UNIX systems
• Smart model discovery: Searches multiple directories for .gguf files
• Partial name matching: Find models without typing complete filenames
• Multi-modal support: Automatically loads companion .mmproj-*.gguf files
• Flexible configuration: Environment variables override INI file settings with platform-appropriate paths
• Fallback defaults: Works out-of-the-box with sensible defaults
• Dry-run capability: Preview commands before execution
• Color-coded output: Enhanced terminal display with syntax highlighting
• Error handling: Clear error messages for missing configurations and models
• Configuration priority: Script directory configs override user configs

Both scripts share identical command syntax and functionality.

• list: List all available .gguf models in configured directories
• <partial_model_name>: Partial name to match against model files (case-insensitive)
• <context_size>: Required - Context window size for the model
• [llama-server args...]: Optional additional arguments passed to llama-server
• [--dry-run]: Preview the command without executing it

llms list

llms <partial_model_name> <context_size>

example:

llms Mistral-Small-3.1-24B-Instruct-2503-UD-Q4 64000

The script automatically detects companion .mmproj-*.gguf files and adds appropriate parameters.

llms <partial_model_name> <context_size> [llama-server args...]

example with custom chat template:

llms GLM-4-32B 24000 --chat-template-file ~/llm/chat-template-chatml.jinja

Preview the command that would be executed without running it:

llms Devstral-Small-2505-UD-Q4 100000 --no-webui --dry-run

1. Environment Variables (always override everything)
2. llms.ini in script directory (highest file priority)
3. llms.ini in user config directory:
   • Windows: %USERPROFILE%\AppData\Local\llms.ini
   • Linux/macOS: ~/.config/llms.ini (or $XDG_CONFIG_HOME/llms.ini)
4. Fallback values in code

All configuration options can be set via environment variables:

The scripts look for llms.ini in these locations (in priority order):

Priority 1: Script Directory

• Same directory as the script (./llms.ini)

Priority 2: User Config Directory

• Windows: %USERPROFILE%\AppData\Local\llms.ini
• Linux/macOS: ~/.config/llms.ini (or $XDG_CONFIG_HOME/llms.ini)

# Model directories (comma-separated paths)
ModelsDirs = C:\path\to\models1,D:\path\to\models2

# Performance settings
CacheTypeK = q8_0
CacheTypeV = q8_0
UbatchSize = 1024
NGpuLayers = 99

# Network settings (can also be set via ENV)
Host = 0.0.0.0
Port = 8080
ApiKey = secret

PowerShell:

$env:LLMS_MODELS_DIRS = "C:\models,D:\ai-models"
$env:LLMS_PORT = 8081
$env:LLMS_N_GPU_LAYERS = 50
llms mistral 32000

Bash:

LLMS_MODELS_DIRS="/home/user/models,/opt/ai-models" \
LLMS_PORT=8080 \
LLMS_N_GPU_LAYERS=50 \
llms mistral 32000

llms.ini:

ModelsDirs = C:\models,D:\ai-models
# or Linux/macOS paths
#ModelsDirs = /home/user/models,/opt/ai-models

UbatchSize = 2048
NGpuLayers = 40

override via environment:

PowerShell:

$env:LLMS_PORT = 8081
llms mistral 32000

Bash:

LLMS_PORT=8081 \
llms mistral 32000

# Model locations
ModelsDirs = C:\Users\username\models,D:\shared-models,E:\large-models

# Performance tuning
CacheTypeK = q8_0
CacheTypeV = q8_0
UbatchSize = 1024
NGpuLayers = 99

# Server settings
Host = 0.0.0.0
Port = 8080
ApiKey = my-secret-key

Both scripts use identical syntax:

llms list
llms <partial_model_name> <context_size> [llama-server args...] [--dry-run]

This project uses Pester v5 for testing the PowerShell script. To run the tests:

1. Ensure Pester is installed:

   Install-Module -Name Pester -Force -Scope CurrentUser

2. Run the tests:

   Invoke-Pester ./llms.tests.ps1

The bash script has been manually tested on macOS and should work on Linux systems. Automated testing for the bash script is planned for future releases.