This is a mono-repository that combines NextJS Frontend and Python FastAPI Backend. The repositories are included as git submodules. Make sure to follow the instrcutions for a correct installation. We are using docker compose to start all required services at once.

  git submodule update --init

  docker-compose up -d --build

Open local web-UI

⚠️ Warning:

Please adapt the docker-compose.yml and the backend Dockerfile to enable CUDA-Support on your Docker-Container. Otherwise the backend may only run on the CPU.

Example: docker-compose.yml

services:
  ...

  backend:
    ...
    environment:
      ...
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
   runtime: nvidia
   ...

Example: backend/Dockerfile

# Use NVIDIA's CUDA base image
FROM nvidia/cuda:11.8.0-cudnn8-runtime-ubuntu22.04

# Set environment variables for CUDA
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES compute,utility

... (add torch/cuda to requirements.txt if )
... (Rest of the Dockerfile)

Another option is to directly run the FastAPI Server on a GPU-capable device - Instructions and connect the frontend manually to the exposed IP